Merge/append CSV files with different headers

Vincent Davis vincent at vincentdavis.net
Mon Mar 24 13:50:07 EDT 2014


I have several csv file I need to append (vertically). They have different
but overlapping headers. For example;
file1 headers ['a', 'b', 'c']
file2 headers ['d', 'e']
file3 headers ['c', 'd']

Is there a better way than this
import csv
def merge_csv(fileList, newFileName):
    allHeaders = set([])
    for afile in fileList:
        with open(afile, 'rb') as csvfilesin:
            eachheader = csv.reader(csvfilesin, delimiter=',').next()
            allHeaders.update(eachheader)
    print(allHeaders)
    with open(newFileName, 'wb') as csvfileout:
        outfile = csv.DictWriter(csvfileout, allHeaders)
        outfile.writeheader()
        for afile in fileList:
            print('***'+afile)
            with open(afile, 'rb') as csvfilesin:
                rows = csv.DictReader(csvfilesin, delimiter=',')
                for r in rows:
                    print(allHeaders.issuperset(r.keys()))
                    outfile.writerow(r)

Vincent Davis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20140324/b5da9c79/attachment.html>


More information about the Python-list mailing list