best way to read a huge ascii file.

[BartC]

On 30/11/2016 16:16, Heli wrote:
> Hi all,
>
>  Writing my ASCII file once to either of pickle or npy or hdf data types and then working afterwards on the result binary file reduced the read time from 80(min) to 2 seconds.

240,000% faster? Something doesn't sound quite right! How big is the 
file now? (The one that had been 40GB of text and contained 100M lines 
or datasets, although file caching is just about a possibility.)

-- 
Bartc