[SciPy-User] HDF4, HDF5, netcdf solutions -- PyNIO/PyNGL or CDAT or ??

[Francesc Alted]

A Tuesday 02 November 2010 04:40:47 Andrew Collette escrigué:
> I think this is partially my fault for not making h5py's purpose
> clearer in the beginning.  From my perspective, h5py is trying to be
> a "native" (as close as possible) Python/NumPy interface to the HDF5
> library, while adding as little as possible.  That means it doesn't
> have any of the advanced indexing features of PyTables, or the
> database metaphor (Francesc, reel me in if I'm getting out of bounds
> here or below).

Well, I recognize that I like to use the "database metaphor" because I 
really think that PyTables is kind of database.  You know, the 
boundaries between a simple format and a database are always fuzzy, but 
definitely, the fact that PyTables can perform fast queries (I mean, 
something typical in RDBMS like ("((lat>.45)|
(lon<.55))&(temp1+temp2)>23)" on very large tables in an efficient way 
(using numexpr & compression internally and, when using Pro, column 
indexes too), the support for metadata and its hierarchical nature, 
makes me speak this way. 

> > Frankly, I think the best rational here is more a matter of trying
> > out the different packages and choose the one you like the most.
> >  This is one of the beauties of free software: easy trying.
> 
> Well said!  I should also point out that Francesc and I have shared
> code and suggestions in the past, which is another great thing about
> free software.  In fact, h5py started off using the PyTables Pyrex
> definitions!  It certainly saved me lots of typing. :)

You are welcome.  I also stole quite a few of code from h5py, as you 
know (the fancy indexing code).  I certainly always thought that h5py is 
a big contribution to HDF5 being more adopted in the scientific arena, 
among other reasons because trusting only in one single package with 
basically only one developer is not really that "trusty" :-)

Cheers,

-- 
Francesc Alted