[Numpy-discussion] fast numpy i/o

[Christopher Barker]

Neal Becker wrote:
>> I'm wondering what are good choices for fast numpy array serialization?
>>
>> mmap: fast, but I guess not self-describing?
>> hdf5: ?
>> pickle: self-describing, but maybe not fast?
>> others?
> 
> I think, in addition, that hdf5 is the only one that easily interoperates with 
> matlab?

netcdf is another option if you want an open standard supported by 
Matlab and the like. I like the netCDF4 package:

http://code.google.com/p/netcdf4-python/

Though is can be kind of slow to write (I have no idea why!)

plain old tofile() should be readable by other tools as well, as long as 
you have a way to specify what is in the file.

> speaking of hdf5, I see:
> 
> pyhdf5io  0.7 - Python module containing high-level hdf5 load and save 
> functions.
> h5py  2.0.0 - Read and write HDF5 files from Python

There is also pytables, which uses HDF5 under the hood.


-Chris





-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov