first, second, etc line of text file

Jay Loden python at jayloden.com
Wed Jul 25 18:38:45 EDT 2007


Grant Edwards wrote:
> On 2007-07-25, Jeff <jeffober at gmail.com> wrote:
> 
>> That might be a memory problem if you are running multiple processes
>> regularly, such as on a webserver.
> 
> I suppose if you did it in parallel 50 processes, you could use
> up 250MB of RAM.  Still not a big deal on many servers.  A
> decent OS will swap regions that aren't being used to disk, so
> it's likely not to be a problem.  

Or, you might be reading from a text file dramatically larger than a 3MB copy of War and Peace. I regularly deal with log files that are often many times that, including some that have been well over a GB or more. Trust me, you don't want to read in the entire file when it's a 1.5GB text file. It's true that many times readlines() will work fine, but there are also certainly cases where it's not acceptable for memory and performance reasons.

-Jay



More information about the Python-list mailing list