csv.DictReader line skipping should be considered a bug?

[Neil Cerutti]

On 2017-12-11, Neil Cerutti <neilc at norwich.edu> wrote:
> On 2017-12-05, Steve D'Aprano <steve+python at pearwood.info> wrote:
>> On Wed, 6 Dec 2017 04:20 am, Jason wrote:
>>> while iterating over two files, which are line-by-line
>>> corresponding. The DictReader skipped ahead many lines
>>> breaking the line-by-line correspondence.
>>
>> Um... this doesn't follow. If they are line-by-line
>> corresponding, then they should skip the same number of blank
>> lines and read the same number of non-blank lines.
>>
>> Even if one file has blanks and the other does not, if you
>> iterate the over the records themselves, they should keep their
>> correspondence.
>>
>> I'm afraid that if you want to convince me this is a buggy
>> design, you need to demonstrate a simple pair of CSV files
>> where the non-blank lines are corresponding (possibly with
>> differing numbers of blanks in between) but the CSV readers get
>> out of alignment somehow.
>
> Preface: I'm not arguing for this to be changed--it obviously
> cannot be at this point, and we know how to work around it when
> it matters--although the current design does make finding the
> erronesou records needlessly harder than it needs to be.
>
> Examine the records that DictReader returns for the following csv
> file and see if you think it still feels obvious and usable.
>
> A,B,C
> a,b,c
> a,b
> a
>
> a,b
> a,b,c
>
> Furthermore, see what DictWriter produces from this program:
>
> with open("wcsv.csv", 'w', newline='') as f:
>     writer = csv.DictWriter(f, fieldnames=('A', 'B', 'C'))
>     writer.writeheader()
>     for rec in (
>             {'A': 'a', 'B': 'b', 'C': 'c'},
>             {'A': 'a', 'B': 'b'},
>             {'A': 'a'},
>             {},
>             {'A': 'a'},
>             {'A': 'a', 'B': 'b'},
>             {'A': 'a', 'B': 'b', 'C': 'c'},):
>         writer.writerow(rec)
>
> DictReader doesn't handle the output of DictWriter in a usable
> and recoverable way.

DOH! Never mind this part of my argument. I'll take some time to
eat these eggs on my face. ;)

-- 
Neil Cerutti