Finding Blank Columns in CSV

Jaydip Chakrabarty chalao.adda at gmail.com
Tue Oct 6 07:24:33 EDT 2015


On Tue, 06 Oct 2015 01:34:17 +1100, Chris Angelico wrote:

> On Tue, Oct 6, 2015 at 1:06 AM, Tim Chase
> <python.list at tim.thechases.com> wrote:
>> That way, if you determine by line 3 that your million-row CSV file has
>> no blank columns, you can get away with not processing all million
>> rows.
> 
> Sure, although that effectively means the entire job is moot. I kinda
> assume that the OP knows that there are some blank columns (maybe lots
> of them). The extra check is unnecessary unless it's actually plausible
> that there'll be no blanks whatsoever.
> 
> Incidentally, you have an ordered_headers list which is the blank
> columns in order; I think the OP was looking for a list of the
> _non_blank columns. But that's a trivial difference, easy to tweak.
> 
> ChrisA

Thanks to you all. I got it this far. But while writing back to another 
csv file, I got this error - "ValueError: dict contains fields not in 
fieldnames: None". Here is my code.

rdr = csv.DictReader(fin, delimiter=',')
header_set = set(rdr.fieldnames)
for r in rdr:
    header_set = set(h for h in header_set if not r[h])
    if not header_set:
        break

for r in rdr:
    data = list(r[i] for i in header_set)

dw = csv.DictWriter(fout, header_set)
dw.writeheader()
dw.writerows(data)

Also, there is difference between len(header_set) and len(data[0].keys). 
Why is so?
Thanks again for all your help.

Thanks.




More information about the Python-list mailing list