Merge/append CSV files with different headers

[Vincent Davis]

Thanks for the feedback.

Vincent Davis
720-301-3003


On Mon, Mar 24, 2014 at 1:44 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Tue, Mar 25, 2014 at 4:50 AM, Vincent Davis <vincent at vincentdavis.net>
> wrote:
> > I have several csv file I need to append (vertically). They have
> different
> > but overlapping headers. For example;
> > file1 headers ['a', 'b', 'c']
> > file2 headers ['d', 'e']
> > file3 headers ['c', 'd']
> >
> > Is there a better way than this
>
> Summary of your code:
>
> 1) Build up a set of all headers used, by opening each file and
> reading the headers.
> 2) Go through each file a second time and write them out.
>
> That seems like the best approach, broadly. You might be able to
> improve it a bit (it might be tidier to open each file once, but since
> you're using two different CSV readers, it'd probably not be), but by
> and large, I'd say you have the right technique. Your processing time
> here is going to be dominated by the actual work of copying.
>
> The only thing you might want to consider is order. The headers all
> have a set order to them, and it'd make sense to have the output come
> out as ['a', 'b', 'c', 'd', 'e'] - the first three from the first
> file, then adding in everything from subsequent files in the order
> they were found. Could be done easily enough by using 'in' and
> .append() on a list, rather than using a set. But if that doesn't
> matter to you, or if something simple like "sort the headers
> alphabetically" will do, then I think you basically have what you
> want.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20140324/edfe7d5b/attachment.html>