[Numpy-discussion] genfromtxt universal newline support

[Derek Homeier]

On 30 Jun 2014, at 04:56 pm, Nathaniel Smith <njs at pobox.com> wrote:

>> A real need, which had also been discussed at length, is a truly performant text IO
>> function (i.e. one using a compiled ASCII number parser, and optimally also a more
>> memory-efficient one), but unfortunately all people interested in implementing this
>> seem to have drifted away (not excluding myself from this)…
> 
> It's possible we could steal some code from Pandas for this. IIRC they
> have C/Cython text parsing routines. (It's also an interesting
> question whether they've fixed the unicode/binary issues, might be
> worth checking before rewriting from scratch...)

Good point, last time I was playing with Pandas it was not any faster, but now a 10x
speedup speaks for itself. Their C engine does not support generic whitespace separators,
but that could probably be addressed in a numpy implementation.

					Derek