[Matrix-SIG] Reading and handling matrices.

[Christos Siopis]

On Fri, 23 Apr 1999, Travis Oliphant wrote:

> I wanted to read arbitrary binary files and so wrote a C-extension module
> that reads arbitrary binary data into Python NumPy arrays.  It turns out,
> I could have accomplished what I wanted to do with the fromstring function
> from the Numeric module and the read method of any file object.

Dear Travis,

Allow me to disagree and point out the usefulness of your numpyio ;-)
I think what you are saying above is true if one does not have to deal
with large (compared to available RAM) data sets. In a recent project, I
had to work with two arrays of 75-100 Mb each. They had to be in memory
simultaneously. Using IDL, I could do that on a 196 Mb machine with
minimal swapping and even on a 128 Mb machine it was bearable. With Python
I had to resort to a 256 Mb machine (since, at one point, it has to hold
the first array, the second array as an array, and also the second array
as a string while conversion takes place). 

IMHO, binary I/O is a frequent task in data analysis and it would make
sense to add a tofile() and a fromfile() method for NumPy arrays, even in
the absence of any efficiency considerations... 

Christos