[melbourne-pug] Problem with reading binary file (different result between MATLAB and Python)

[John Machin]

On 12/01/2009 12:08 AM, Hani Zahiri wrote:
> Hi folks,

> I am trying to translate one of my MATLAB scripts to Python and I am 
> experiencing a strange problem (at least to me!) and I am desperetly 
> looking for help. The binary file is a raw binary containing header 
> information (first 720 bytes) following by radar data. For better 
> illustration and using python basic functions, first 800 bytes of the 
> file is look like this:

>  >>> fid = open(“file_name”,”rb”)
> 
>  >>> fid.read(800)
> 
> '\x00\x00\x00\x012\xc0\x12\x12\x00\x00\x02\xd0A   CEOS-SAR-CCT A A 
[snip 700+ bytes of irrelevant stuff]

> And now the problem is:
> 
> If I read the file in MATLAB, let say to find out length of header part, 
> I will get the correct answer:
> EDU>> fid=fopen('file_name','r','b');
> EDU>> fseek(fid,8,'bof');
> EDU>> fread(fid,1,'uint32')
> ans =
>    720
> However if I read this in python I am keep getting this wrong:
>  >>> fid.seek(8)
>  >>> scipy.fromfile(fid,'uint32',1)
> array([3489792000], dtype=uint32)
> 
> I have almost tried every Scipy and Numpy classes with no result. I need 
> a quick answer to this and I appreciate if anybody can help me with this 
> problem.

Hello, Hani,

Have you asked anywhere else? Flattered by the implication that 
melbourne-pug is the fount of all knowledge, somewhat dismayed by the 
thought that we're expected [in some sense] to be thinking Python at 
midnight on a Sunday :-)

For future reference, news:comp.lang.python covers all timezones.

Not being a numpy/scipy person, I'd use struct.unpack:
| >>> hex(720)
| '0x2d0'
| >>> guff = '\x00\x00\x00\x012\xc0\x12\x12\x00\x00\x02\xd0A   CEOS-SAR-CCT'
| >>> guff[8:12]
| '\x00\x00\x02\xd0'
| >>> import struct
| >>> struct.unpack('>I', guff[8:12])
| (720,)
| >>> struct.unpack('<I', guff[8:12])
| (3489792000L,)
| >>>

Looks like the problem is inability to distinguish the big end from the 
little end :-)

I'd suggest asking on the numpy/scipy mailing-list -- I'd be very 
surprised if there wasn't already a way of reading MATLAB files in 
Python at a higher level than using seek/fromfile/etc.

HTH,
John