[Numpy-discussion] saving groups of numpy arrays to disk

[Paul Anton Letnes]

On 25. aug. 2011, at 23.49, David Warde-Farley wrote:

> On 2011-08-25, at 2:42 PM, Chris.Barker wrote:
> 
>> On 8/24/11 9:22 AM, Anthony Scopatz wrote:
>>>   You can use Python pickling, if you do *not* have a requirement for:
>> 
>> I can't recall why, but it seem pickling of numpy arrays has been 
>> fragile and not very performant.
>> 
>> I like the npy / npz format, built in to numpy, if you don't need:
>> 
>>>   - access from non-Python programs
> 
> While I'm not aware of reader implementations for any other language, NPY is a dirt-simple and well-documented format designed by Robert Kern, and should be readable without too much trouble from any language that supports binary I/O. The full spec is at
> 
> https://github.com/numpy/numpy/blob/master/doc/neps/npy-format.txt
> 
> It should be especially trivial to read arrays of simple scalar numeric dtypes, but reading compound dtypes is also doable.
> 
> For NPZ, use a standard zip file reading library to access individual files in the archive, which are in .npy format (or just unzip it by hand first -- it's a normal .zip file with a special extension).
> 
> David

Out of curiosity: is the .npy format guaranteed to be independent of architecture (endianness and similar issues)?

Paul