[Numpy-discussion] np.loadtxt : yet a new implementation...

[Pierre GM]

On Dec 3, 2008, at 12:48 PM, Christopher Barker wrote:

> Pierre GM wrote:
>> I can try, but in that case, please write me a unittest, so that I
>> have a clear and unambiguous idea of what you expect.
>
> fair enough, though I'm not sure when I'll have time to do it.

Oh, don;t worry, nothing too fancy: give me a couple lines of input  
data and a line with what you expect. Using Ryan's recent example:

 >>>f = StringIO('stid stnm relh tair\nnrmn 121 45 9.1')
 >>> test = loadtxt(f, usecols=('stid', 'relh', 'tair'), names=True,  
dtype=None)
 >>> control=array(('nrmn', 45, 9.0999999999999996),
     				 dtype=[('stid', '|S4'), ('relh', '<i8'), ('tair', '<f8')])

That's quite enough for a test.

> I do wonder if anyone else thinks it would be useful to have multiple
> delimiters as an option. I got the idea because with fromfile(), if  
> you
> specify, say ',' as the delimiter, it won't use '\n', only  a comma,  
> so
> there is no way to quickly read a whole bunch of comma delimited  
> data like:
>
> 1,2,3,4
> 5,6,7,8
> ....
>
> so I'd like to be able to say to use either ',' or '\n' as the  
> delimiter.

I'm not quite sure I follow you.
Do you want to delimiters, one for the field of a record (','), one  
for the records ("\n") ?



>
> However, if I understand loadtxt() correctly, it's handling the new
> lines separately anyway (to get a 2-d array), so this use case isn't  
> an
> issue. So how likely is it that someone would have:
>
> 1  2  3, 4, 5
> 6  7  8, 8, 9
>
> and want to read that into a single 2-d array?

With the current behaviour, you gonna have
[("1 2 3", 4, 5), ("6 7 8", 8, 9)] if you use "," as a delimiter,
[(1,2,"3,","4,",5),(6,7,"8,","8,",9)] if you use " " as a delimiter.

Mixing delimiter is doable, but I don't think it's that a good idea.  
I'm in favor of sticking to one and only field delimiter, and the  
default line spearator for record delimiter. In other terms, not  
changing anythng.