[PYTHON MATRIX-SIG] Statistics and file formats

[Phil Austin]

>>>>> "David" == David Ascher <da@maigret.cog.brown.edu> writes:


    David> 2) I'm considering using netCDF as the file format for some
    David> tools I'm building.  There are alternatives, however, such
    David> as HDF.  My interest in these is a standardized way of
    David> having self-describing data, not the ability to handle very
    David> large datasets (that doesn't happen with my data).  Is
    David> there a consensus out there which I should be aware of?
    David> I've also looked a little at some of the tools that Paul
    David> mentioned, such as SILO/SLIDE.  I have no good basis on
    David> which to make one decision or another (except that there is
    David> a netcdf module out there, although it appears to do 1-D
    David> data only for now).


We have to support both HDF and netcdf interfaces for satellite
and aircraft data, and my impressions of the two are:

1) HDF is more flexible (Vdata allows pointer-like structures
that netCDF doesn't)

2) Netcdf is much better documented, and there are more browsers,
etc., written for it. (as evidenced by the respective home pages).
Netcdf also has Splus, Matlab, C++, perl, and java (this one
read-only) interfaces.

3) NASA has adopted HDF in a big way.  They have been funding
convergence of the two formats via a netCDF interface in HDF, at a
cost in (HDF) efficiency.  NOAA and NCAR (obviously) prefer netCDF.
There is a new netCDF version being tested now that makes some
significant API changes (and doubles the speed of netCDF)--I don't
know whether the NCSA HDF developers are committed to tracking these
changes.

Since we are doing plain packed n-dimensional arrays we have 
focussed on netCDF.  Kyle Schalm, a co-op student working with me on
Python image processing, is currently adding Numeric arrays to William
Noon's package, he should have a beta version in a week or two.

Regards, Phil


Phil Austin		INTERNET: phil@geog.ubc.ca
(604) 822-2175		FAX:	  (604) 822-6150

http://www.geog.ubc.ca/~phil
Associate Professor
Atmospheric Sciences Programme
Geography #217
University of British Columbia
1984 W Mall
Vancouver, BC  V6T 1Z2
CANADA

=================
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================