[SciPy-dev] Binary i/o package

[Ravikiran Rajagopal]

On Sunday 03 June 2007 4:42:04 pm Erin Sheldon wrote:
> This package fills the niche and is the backbone of such systems.  And
> it is a small chunk of code.  You can extract what you want from the
> file and store it directly into a numpy array in the most efficient
> manner possible.

Apologies for the slow reply, but only now did I find time to go through your 
code. I agree with you that this is a pretty useful piece of code. However, 
the functionality offered by your code, IMHO, should be split into two parts:
  - readbinarray / writebinarray /skipfields
  - readheader / writeheader

Possible prototypes would be as follows:
  readbinarray( fid, fieldtuple, columns, lines, headerskip=0 )
  writebinarray( fid, fieldtuple, columns, lines )
  skipfields( fid, fieldtuple, lines )

"fieldtuple" describes the structure of each record. This set of functions 
would make your code the equivalent of read_array and write_array without 
involving "self-documentation" of binary files. This allows arbitrary headers 
and arbitrary parsers of the header data.

The second set of functions provides default methods for reading/writing 
headers. Combining these orthogonal functions gives the current interface.

I would be very interested in seeing the first part in scipy.io. If no one 
else is interested in having a binary equivalent of read_array/write_Array in 
scipy, something like this is a perfect candidate for a scikit.

Regards,
Ravi