python: ascii read
Robert Kern
rkern at ucsd.edu
Thu Sep 16 08:05:03 EDT 2004
Sebastian Krause wrote:
> Hello,
>
> I tried to read in some large ascii files (200MB-2GB) in Python using
> scipy.io.read_array, but it did not work as I expected. The whole idea
> was to find a fast Python routine to read in arbitrary ascii files, to
> replace Yorick (which I use right now and which is really fast, but not
> as general as Python). The problem with scipy.io.read_array was, that it
> is really slow, returns errors when trying to process large files and it
> also changes (cuts) the files (after scipy.io.read_array processed a 2GB
> file its size was only 64MB).
>
> Can someone give me hint how to use Python to do this job correctly and
> fast? (Maybe with another read-in routine.)
What kind of data is it? What operations do you want to perform on the
data? What platform are you on?
Some of the scipy.io.read_array behavior that you see look like bugs. We
would greatly appreciate it if you were to send a complete bug report to
the scipy-dev mailing list. Thank you.
--
Robert Kern
rkern at ucsd.edu
"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
More information about the Python-list
mailing list