append one file to another

[Steven D'Aprano]

On Tue, 12 Jul 2005 06:47:50 -0700, b83503104 at yahoo.com wrote:

> Hi,
> 
> I want to append one (huge) file to another (huge) file.

What do you call huge? What you or I think of as huge is not necessarily
huge to your computer.

> The current
> way I'm doing it is to do something like:
> 
> infile = open (infilename, 'r')
> filestr = infile.read()
> outfile = open(outfilename, 'a')
> outfile.write(filestr)
> 
> I wonder if there is a more efficient way doing this?

Why? Is it not working? Is it too slow? Does it crash your computer?

If you have any expectation that you code needs to run under Windows, or
cross-platform, or contains binary data, you should open your files in
binary mode:

infile = open(infilename, 'rb')
outfile = open(outfilename, 'ab')

For raw copying, you should probably use binary mode even if they just
contain text. Better safe than sorry...

Then, if you are concerned that the files really are huge, that is, as big
or bigger than the free memory your computer has, read and write them in
chunks:

data = infile.read(64)  # 64 bytes at a time is a bit small...
outfile.write(data)

Instead of 64 bytes, you should pick a more realistic figure, which will
depend on how much free memory your computer has. I suppose a megabyte is
probably reasonable, but you will need to experiment to find out.

Then when you are done, close the files:

infile.close()
outfile.close()

This is not strictly necessary, but it is good practice. If your program
dies, the files may not be closed properly and you could end up losing
data.


-- 
Steven.