[SciPy-User] Maximum file size for .npz format?

[Christopher Barker]

Nathaniel Smith wrote:
> On Sun, Mar 14, 2010 at 5:24 PM, Christopher Barker
> <Chris.Barker at noaa.gov> wrote:
>> It might be nice to build something on libbzip2 -- It looks like the
>> license is right, it's got good compression qualities, supports 64 bit,
>> and it getting pretty widely used.
> 
> If we're going to pull in a new C library for this purpose, then maybe
> we should just use libhdf5? :-).

except that hdf is one big honking pain in the butt to build.

Robert's right, bzip isn't a great option anyway -- it can do multiple 
files, but it just concatenates them, and doesn't appear to provide an 
index. I used to use afio, which would zip each file first, then put 
them all in an archive -- I liked that approach. We could, of course, 
build something like that with bzip, but it looks like python's zip will 
work for >= 2.6, so no need for something new.

-Chris






-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov