[SciPy-dev] Boost vs CXX and SCXX

[Francesc Alted]

On Fri, Sep 13, 2002 at 04:49:47PM -0600, Chuck Harris wrote:
> > 
> > I've been working for a while in a wrapper for HDF5 format in 
> > Python and I'm
> 
> I would be very interested in that. How far along are you?

Good!. At the moment I'm mainly interested in implementing tables based on
compound types (C structs). The package (probably I'll call it PyTables, but
that's still unsure) can do now:

- Create tables and feed them with an unlimited number of records

- Supports data buffering (can make writes and reads orders of magnitude
faster) 

- Read the record tables from objects acting as iterators 

- Create a hierarchical structure with the tables

In the next week I plan to support also Numeric arrays, and document a bit
the package. If I have time I'll include some tests. Hopefully, in the end
of September, I'll make available an alpha release package for public
evaluation. I want to release early the code in order to get feedback from
the scientific community.

Here you have a small example of what can be done right now:

from H5File import *
from Numeric import *

totalrows = 10

file = H5File(filename = 'h5_table.h5')   # Create a new HDF5 file
b = file.make_instance()                  # Read the file metadata

# Create a table
b.tuple1 = c = H5Table(varnames = ('var1', 'var2', 'var3'),
		       fmt = '3sid', tableTitle = "Table 1",
		       compress = 0, expectedrows = totalrows)

# Fill the table
for i in xrange(totalrows):
    c.commitBuffered(str(i), i, 12.1e40)
c.flush()                    # Flush the buffer table
b.direcname = d = H5Dir()    # Create a new directory on file

# Create another table in the new directory
d.tuple1 = c = H5Table(varnames = ('var1', 'var2', 'var3', 'var4'),
		       fmt = '3sidd', tableTitle = "Table 1",
		       compress = 0, expectedrows = totalrows)

# Fill the second table
for i in xrange(totalrows):
    c.commitBuffered(str(i), i, 12.1e40, 12.1e40 * i)
c.flush()                    # Flush the buffer table

# Example of tuple selection
numarray = array([ tupla[1] for tupla in c.readAllTable() if tupla[1] < 20 ])
print "Total selected records ==> ", len(numarray)

# Close the HDF5 file
file.close()

The h5ls output over this file can be seen here:
$ h5ls -rd h5_table.h5
/h5_table.h5/direcname   Group
/h5_table.h5/direcname/tuple1 Dataset {10/Inf}
    Data:
        (0) {"0", 0, 1.21e+41, 0}, {"1", 1, 1.21e+41, 1.21e+41},
	(2) {"2", 2, 1.21e+41, 2.42e+41}, {"3", 3, 1.21e+41, 3.63e+41},
	(4) {"4", 4, 1.21e+41, 4.84e+41}, {"5", 5, 1.21e+41, 6.05e+41},
        (6) {"6", 6, 1.21e+41, 7.26e+41}, {"7", 7, 1.21e+41, 8.47e+41},
	(8) {"8", 8, 1.21e+41, 9.68e+41}, {"9", 9, 1.21e+41, 1.089e+42}
/h5_table.h5/tuple1      Dataset {10/Inf}
    Data:
        (0) {"0", 0, 1.21e+41}, {"1", 1, 1.21e+41}, {"2", 2, 1.21e+41},
        (3) {"3", 3, 1.21e+41}, {"4", 4, 1.21e+41}, {"5", 5, 1.21e+41},
        (6) {"6", 6, 1.21e+41}, {"7", 7, 1.21e+41}, {"8", 8, 1.21e+41},
        (9) {"9", 9, 1.21e+41}

As always, suggestions or support of any kind is welcomed.


-- 
Francesc Alted                            PGP KeyID:      0x61C8C11F
OpenLC microkernel benchmarking project: http://www.openlc.org
Public PGP key available: http://www.openlc.org/falted_at_openlc.asc
Key fingerprint = 1518 38FE 3A3D 8BE8 24A0  3E5B 1328 32CC 61C8 C11F