[Numpy-discussion] genfromtxt converter question

[Derek Homeier]

On 18 Jun 2011, at 04:48, gary ruben wrote:

> Thanks guys - I'm happy with the solution for now. FYI, Derek's
> suggestion doesn't work in numpy 1.5.1 either.
> For any developers following this thread, I think this might be a nice
> use case for genfromtxt to handle in future.

Numpy 1.6.0 and above is handling it in the present.

> As a corollary of this problem, I wonder whether there's a
> human-readable text format for complex numbers that genfromtxt can
> currently easily parse into a complex array? Having the hard-coded
> value for the number of columns in the converter and the genfromtxt
> call goes against the philosophy of the function's ability to form an
> array of shape matching the input layout.

genfromtxt and (for regular data such as yours) loadtxt can also parse  
the standard
'1.0+3.14j ...' format practically automatically as long as you  
specify 'dtype=complex'
(and appropriate delimiter, if required). loadtxt does not handle  
dtype=np.complex64
or np.complex256 at this time due to a bug in the default converters;  
I have just created
a patch for that.
In principle genfromtxt and loadtxt in numpy 1.6 can also handle cases  
similar to your input,
but you need to change the tuples at least to contain no spaces -
'( -1.4249, 1.7330)' -> '(-1.4249,+1.7330)'
or additionally insert another delimiter like ';' - otherwise it's  
hopeless to correctly infer
the number of columns.
With such input, np.genfromtxt(a, converters=cnv, dtype=complex) works  
as expected,
and I have also just created a patch that would allow users to more  
easily specify a
default converter for all input data rather than constructing one for  
every row.

A wider-reaching automatic detection of complex formats might be  
feasible for
genfromtxt, but I'd suspect it could create quite some overhead, as I  
can think of at least
two formats that should probably be considered - tuples as above, and  
pairs of
'real,imag' without parentheses. But feel free to create a ticket if  
you think this would be
an important enhancement.

Cheers,
								Derek