[Numpy-discussion] Bug in genfromtxt with usecols and converters

[Derek Homeier]

Hi Adrian,

> I tried to load data from a csv file into numpy using genfromtxt. I need
> only a subset of the columns and want to apply some conversions to the
> data. attached is a minimal script showing the error.
> In brief, I want to load columns 1,2 and 4. But in the converter
> function for the 4th column, I get the 3rd value. The issue does not
> occur if I also load the 3rd column.
> Did I somehow misunderstand how the function is supposed to work or is
> this indeed a bug?

not sure whether to call it a bug; the error seems to arise before reading any actual data
(even on reading from an empty string); when genfromtxt is checking the filling_values used
to substitute missing or invalid data it is apparently testing on default testing values of 1 or -1
which your conversion scheme does not know about. Although I think it is rather the user’s
responsibility to provide valid converters, probably the documentation should at least be
updated to make them aware of this requirement.
I see two possible fixes/workarounds:

provide an keyword argument filling_values=[0,0,'1:1’]
or add the default filling values to your relEnum dictionary, e.g.  { … '-1':-1, '1':-1}

Could you check if this works for your case?

HTH,
						Derek