Deleting lines from a file

[Tim Chase]

> I need to write a program which reads an external text file. Each time
> it reads, then it needs to delete some lines, for instance from second
> line to 55th line. The file is really big, so what do you think is the
> fastest method to delete specific lines in a text file ?

Generally, with files that are "really big", you either want to 
edit them in place (which takes a database-type structure), or 
you need to stream through the file a line/window at a time, 
dumping the output to a temporary output file.  The *nix tool for 
this job is sed:

   sed '2,55d' infile.txt > outfile.txt

(it doesn't get much more consise than this).

That's about the same as the following in Python

   out = file('outfile.txt', 'w')
   for i, line in enumerate(file('infile.txt')):
     if 1 < i < 54: continue
     out.write(line)
   out.close()

If you want it "in place", sed will do the output file and 
renaming for you with

   sed -i '2,55d' file.txt

whereas in the Python variant, you'd have to then use the 
os.rename call to move outfile.txt to infile.txt

The Python version is a bit more flexible, as you can add other 
logic to change your bounds.  Not that sed isn't flexible, but it 
starts getting unreadible very quickly as logic grows.

-tkc