simultaneous reading and writing a textfile

[Peter Otten]

Marco Herrn wrote:

> I have a text file with some lines in it.
> Now I want to iterate over this file and exchange some lines with some
> others. I tried this approach:

> This should inspect the file and find the first line, that can't be
> split into two parts (that means, which has only one word in it).
> This line should be exchanged with a line that contains some more
> info.
> 
> Unfortunately (or how better python programmers than I am would say,
> "of course") this doesn't work. The line is exchanged, but also some
> more lines.
> 
> Now how can I achieve, what I want? Really exchange one line with
> another, regardless of their length. Is this possible? If this is not
> possible, then what would be the best approach to do this?

A file is exposed as a sequence of bytes. You can only exchange a string of
bytes with the same number of (different) bytes. Inserting or removing
bytes means you have to rewrite the file from the insertion/deletion point
onwards. Usually you don't bother and just rewrite the whole file.

> I do not want to read the whole file, exchange the line in memory and
> then write the whole file. This would be quite slow with large files.

Here's a simple approach that can deal with large files:

# does not work on windows
BLOCKSIZE = 2**20
infile = open(filename)
os.unlink(filename) # rename if you want to play it safe
outfile = open(filename, "w")
lines = iter(infile.readline, "")
# copy one line at a time
for line in lines:
    parts = line.split(None, 1)
    if len(parts) == 1:
        outfile.write("%s %s\n" % (parts[0], mac))
        break
    outfile.write(line)
# after the insertion point proceed with larger steps
for block in iter(lambda: infile.read(BLOCKSIZE), ""):
    outfile.write(block)
infile.close()
outfile.close()

When this becomes too slow you should consider a database.

Peter