[SciPy-user] Reading in data as arrays, quickly and easily?

[Eric Jonas]

Hello! I'm trying to read in large chunks of binary data as arrays, but
the file formats are complex enough that there is lots of junk that
needs to be skipped over. I have a functioning datafile object in python
with a read(N) method that returns the next N data points in the file,
doing the various raw manipulations, endian conversions, and the like
internally. 

The problem is that it's really really slow. I've spent most of today
playing with boost.python, weave, raw numeric and numarray integration,
and the like. And yet I still can't figure out what I should -really- be
using.

Does anyone have any suggestions for approaches that have worked for
them? Ideally I'd love to read in my data and perform really basic
preprocessing in c++ and then return the resulting arrays to manipulate
using python. I'd rather not spend the weekend reinventing the wheel...

Thanks!
			...Eric