Joining Big Files

[Paul McGuire]

On Aug 25, 4:57 am, mosscliffe <mcl.off... at googlemail.com> wrote:
> I have 4 text files each approx 50mb.
>

<yawn> 50mb? Really?  Did you actually try this and find out it was a
problem?

Try this:
import time

start = time.clock()
outname = "temp.dat"
outfile = file(outname,"w")
for inname in ['file1.dat', 'file2.dat', 'file3.dat', 'file4.dat']:
    infile = file(inname)
    outfile.write( infile.read() )
    infile.close()
outfile.close()
end = time.clock()

print end-start,"seconds"

For 4 30Mb files, this takes just over 1.3 seconds on my system.  (You
may need to open files in binary mode, depending on the contents, but
I was in a hurry.)

-- Paul