[Numpy-discussion] genfromtxt converter question

[Olivier Delalleau]

For the hardcoded part, you can easily read the first line of your file and
split it with the same delimiter to know the number of columns.
It's sure it'd be best to be able to be able to skip this part, but you
don't need to hardcode this number into your code at least.
Something like:
n_cols = len(open('myfile.txt').readline().split(delimiter))

-=- Olivier

2011/6/17 gary ruben <gruben at bigpond.net.au>

> Thanks guys - I'm happy with the solution for now. FYI, Derek's
> suggestion doesn't work in numpy 1.5.1 either.
> For any developers following this thread, I think this might be a nice
> use case for genfromtxt to handle in future.
> As a corollary of this problem, I wonder whether there's a
> human-readable text format for complex numbers that genfromtxt can
> currently easily parse into a complex array? Having the hard-coded
> value for the number of columns in the converter and the genfromtxt
> call goes against the philosophy of the function's ability to form an
> array of shape matching the input layout.
>
> Gary
>
> On Sat, Jun 18, 2011 at 7:24 AM, Derek Homeier
> <derek at astro.physik.uni-goettingen.de> wrote:
> > On 17.06.2011, at 11:01PM, Olivier Delalleau wrote:
> >
> >>> You were just overdoing it by already creating an array with the
> converter, this apparently caused genfromtxt to create a structured array
> from the input (which could be converted back to an ndarray, but that can
> prove tricky as well) - similar, if you omit the dtype=None. The following
> >>>
> >>> cnv = dict.fromkeys(range(4), lambda x: complex(*eval(x)))
> >>> b = np.genfromtxt(a,converters=cnv, dtype=None, delimiter=18,
> usecols=range(4))
> >>>
> >>> directly produces a shape(4,4) complex array for me (you may have to
> apply an .astype(np.complex64) afterwards if so desired).
> >>>
> >>> BTW I think this is an interesting enough case of reading non-trivially
> structured data that it deserves to appear on some examples or cookbook
> page.
> >>>
> >>> HTH,
> >>>                                                                Derek
> >>>
> >>> _______________________________________________
> >>> NumPy-Discussion mailing list
> >>> NumPy-Discussion at scipy.org
> >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >>>
> >> I had tried that as well and it doesn't work with numpy 1.4.1 (I get an
> object array). It may have been fixed in a later version.
> >
> > OK, I was using the current master from github, but it works in 1.6.0 as
> well. I still noticed some differences between loadtxt and genfromtxt
> behaviour, e.g. where loadtxt would be able to take a string from the
> converter and automatically convert it to a number, whereas in genfromtxt
> the converter still had to include the float() or complex()...
> >
> >                                                Derek
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110618/8a954821/attachment.html>