[Numpy-discussion] convert csv file into recarray without pre-specifying dtypes and variable names

[Torgil Svensson]

Nice,

I haven't gone through all details. That's a nice new "missing"
feature, maybe all instances where we can't find a conversion should
be "nan". A few comments:

1. The "load_search" functions contains all memory/performance
overhead that we wanted to avoid with the fromiter function. Does this
mean that you no longer have large text-files that change sting
representation in the columns (aka "0" floats) ?

2.	ident=" "*4
This has the same spelling error as in my first compile try .. it was
meant to be "indent"

3.	types = list((i,j) for i, j in zip(varnm, types2))
Isn't this the same as "types = zip(varnm, types2)" ?

4.		return N.fromiter(iter(reader),dtype = types)
Isn't "reader" an iterator already? What does the "iter()" operator do
in this case?

Best regards,

//Torgil


On 7/18/07, Vincent Nijs <v-nijs at kellogg.northwestern.edu> wrote:
>
>  I combined some of the very useful comments/code from Tim and Torgil and
> came-up with the attached program to read csv files and convert the data
> into a recarray. I couldn't use all of their suggestions because, frankly, I
> didn't understand all of them :)
>
>  The program use variable names if provided in the csv-file and can
> auto-detect data types. However, I also wanted to make it easy to specify
> data types and/or variables names if so desired. Examples are at the bottom
> of the file. Comments are very welcome.
>
>  Thanks,
>
>  Vincent
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
>