[Numpy-discussion] striding through arbitrarily large files

[Sturla Molden]

RayS <rays at blue-cove.com> wrote:

> Thanks Daniele, I'll be trying mmap with Python64. With 32 bit the 
> mmap method throws MemoryError with 2.5GB files...
> The idea is that we allow the users to inspect the huge files 
> graphically, then they can "zoom" into regions of interest and then 
> load a ~100 MB en block for the usual spectral analysis.

Transfer the file to Pytables, and you can zoom as you like.

Or use a recent Python so mmap actually has an offset argument.

Sturla