[Numpy-discussion] reading big-endian uint16 into array on little-endian machine

[Francesc Alted]

A Thursday 17 June 2010 16:29:54 greg whittier escrigué:
> I have files (from an external source) that contain ~10 GB of
> big-endian uint16's that I need to read into a series of arrays.  What
> I'm doing now is
> 
> import numpy as np
> import struct
> 
> fd = open('file.raw', 'rb')
> 
> for n in range(10000)
>     count = 1024*1024
>     a = np.array([struct.unpack('>H', fd.read(2)) for i in range(count)])
>     # do something with a
> 
> It doesn't seem very efficient to call struct.unpack one element at a
> time, but struct doesn't have an unpack_farray version like xdrlib
> does.  I also thought of using the array module and .byteswap() but
> the help says it only work on 4 and 8 byte arrays.

Maybe is a problem with docs.  This works for me:

In [23]: import numpy as np

In [24]: a = np.arange(10)

In [25]: a
Out[25]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [26]: a = np.arange(10, dtype='i2')

In [27]: a.byteswap()
Out[27]: array([   0,  256,  512,  768, 1024, 1280, 1536, 1792, 2048, 2304], 
dtype=int16)

-- 
Francesc Alted