From H.Zahiri at curtin.edu.au Sun Jan 11 13:55:06 2009 From: H.Zahiri at curtin.edu.au (Hani Zahiri) Date: Sun, 11 Jan 2009 21:55:06 +0900 Subject: [sapug] Problem with reading binary file (different result between MATLAB and Python) Message-ID: <82200558F6DE2C479D381D3000D1551C04DC9013@EXMSK1.staff.ad.curtin.edu.au> Hi folks, I am trying to translate one of my MATLAB scripts to Python and I am experiencing a strange problem (at least to me!) and I am desperetly looking for help. The binary file is a raw binary containing header information (first 720 bytes) following by radar data. For better illustration and using python basic functions, first 800 bytes of the file is look like this: >>> fid = open("file_name","rb") >>> fid.read(800) '\x00\x00\x00\x012\xc0\x12\x12\x00\x00\x02\xd0A CEOS-SAR-CCT A A 1.00 1AL1 PSRBIMOP FSEQ 1 4FTYP 5 4FLGT 9 4 18432 88220 32 2 8 1 18432 0 10976 0 0 0BSQ 1 1 412 87808 0 13 4PB 49 2PB 45 4PB 21 4PB 29 4PB 97 4PB COMPLEX*8 C*8 0 0 \x00\x00\x00\x022\n\x12\x14\x00\x01X\x9c\x00\x00\x00\x01\x00\x00\x00\x01 \x00\x00\x00\x00\x00\x00*\xe0\x00\x00\x00\x00\x00\x00\x00\x00\ ... x00\x00\x07\xd6\x00\x00\x00\x8d\x02\xdd\xfe\x95\x00\x01\x00\x00\x00\x00\ x00\x00\x00\x1c\xae\x93\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ix ... \x00\x00\x00\x00I}.\xd0' And now the problem is: If I read the file in MATLAB, let say to find out length of header part, I will get the correct answer: EDU>> fid=fopen('file_name','r','b'); EDU>> fseek(fid,8,'bof'); EDU>> fread(fid,1,'uint32') ans = 720 However if I read this in python I am keep getting this wrong: >>> fid.seek(8) >>> scipy.fromfile(fid,'uint32',1) array([3489792000], dtype=uint32) I have almost tried every Scipy and Numpy classes with no result. I need a quick answer to this and I appreciate if anybody can help me with this problem. Cheers, Hani From dt-sapug at handcraftedcomputers.com.au Sun Jan 11 21:21:46 2009 From: dt-sapug at handcraftedcomputers.com.au (Daryl Tester) Date: Mon, 12 Jan 2009 06:51:46 +1030 Subject: [sapug] Problem with reading binary file (different result between MATLAB and Python) In-Reply-To: <82200558F6DE2C479D381D3000D1551C04DC9013@EXMSK1.staff.ad.curtin.edu.au> References: <82200558F6DE2C479D381D3000D1551C04DC9013@EXMSK1.staff.ad.curtin.edu.au> Message-ID: <496A54DA.70707@handcraftedcomputers.com.au> Hani - just letting you know that this email to sapug came from an unsubscribed address, so you may not see the responses that are sent back solely to the mailing list. Hani Zahiri wrote: > I am trying to translate one of my MATLAB scripts to Python and I am > experiencing a strange problem (at least to me!) and I am desperetly > looking for help. The binary file is a raw binary containing header > information (first 720 bytes) following by radar data. For better > illustration and using python basic functions, first 800 bytes of the > file is look like this: ... > '\x00\x00\x00\x012\xc0\x12\x12\x00\x00\x02\xd0A CEOS-SAR-CCT A A 1.00 ... > ans = > > 720 ... >>>> fid.seek(8) > >>>> scipy.fromfile(fid,'uint32',1) > > array([3489792000], dtype=uint32) I don't know anything about scipy, but this is just an endian problem. Given the binary string you're trying to read (which I'm doing some interpreting here, as the seek doesn't quite marry up with the initial string above): >>> s = '\x00\x00\x02\xd0' >>> import struct >>> print struct.unpack('>> print struct.unpack('>I', s) (720,) It appears you need to unpack the data big-endian to get the number that you want. How Scipy copes with endian-ness is left as an exercise to the recipient. :-) -- Regards, Daryl Tester "Oh Christmas tree, oh Christmas tree! From hell's heart I stab at thee." -- A very Kaaahn! Christmas From robfcarr at gmail.com Thu Jan 22 03:35:19 2009 From: robfcarr at gmail.com (R C) Date: Thu, 22 Jan 2009 13:05:19 +1030 Subject: [sapug] Any Python opportunities in Adelaide? Message-ID: All, I just moved to Adelaide from the US (Seattle, if you are wondering), and I am interested in any suggestions anyone might have for where to look for jobs that use Python. My most recent position involved writing automation code in Python to test network application software, so I am reasonably competent in Python but not what anyone would consider an expert. Thanks in advance, Rob Carr From dt-sapug at handcraftedcomputers.com.au Thu Jan 22 04:02:39 2009 From: dt-sapug at handcraftedcomputers.com.au (Daryl Tester) Date: Thu, 22 Jan 2009 13:32:39 +1030 Subject: [sapug] Any Python opportunities in Adelaide? In-Reply-To: References: Message-ID: <4977E1CF.6010606@handcraftedcomputers.com.au> R C wrote: > I just moved to Adelaide from the US (Seattle, if you are wondering), and I > am interested in any suggestions anyone might have for where to look for > jobs that use Python. My most recent position involved writing automation > code in Python to test network application software, so I am reasonably > competent in Python but not what anyone would consider an expert. Warning: my opinion only, may not mesh with reality. May be taken in jest, but not to be ingested. There's stuff that comes up sporadically via the job hunters, but generally in Adelaide Python skillz seem to be an adjunct to some primary attribute, like sys adminning or assassin (repeating myself there). There has been a games house and video effects outfit advertising in the past though. Most of the ISPs here seem to run Perl (although one is dabbling in Erlang, or trying to. Drat, I don't think he's on this list so he may not bite :-). Well, I know of one ISP using Python, but they have me, so it probably wasn't their choice ... You could carve a niche for yourself though. Currently I'm writing a Windows service and GUI client for a customer using Python and py2exe, and they'll be none the wiser (apart from I've already told them :-). And I'm quite impressed with how far py2exe has come - it's ability to pack the entire interpreter, library and miscellaneous DLLs into a single EXE without having to unpack anything was pretty much a deal breaker for me using Python on this project. I should get another meetup together sometime - one every two years doesn't seem often enough, although it's fitting in with my free schedule nicely ... -- Regards, Daryl Tester "Oh Christmas tree, oh Christmas tree! From hell's heart I stab at thee." -- A very Kaaahn! Christmas From robfcarr at gmail.com Fri Jan 23 03:45:07 2009 From: robfcarr at gmail.com (R C) Date: Fri, 23 Jan 2009 13:15:07 +1030 Subject: [sapug] Okay, let's redirect. Message-ID: All, Having asked about specific Python opportunities yesterday, I thought I would ask a somewhat broader question about networking companies. I know some local companies, including internode and Expand Networks/Netpriva, but does anyone know of other networking companies in the Adelaide area that might have use for someone with a background in Network Support/Testing/Test Automation? Thanks, Robert From steve at adam.com.au Sat Jan 24 05:52:43 2009 From: steve at adam.com.au (stephen white) Date: Sat, 24 Jan 2009 15:22:43 +1030 Subject: [sapug] Pluggable types Message-ID: This is about pluggable types: http://bracha.org/pluggable-types.pdf But has the best criticism of type systems that I've seen. Yes, mandatory typing does make things more brittle, and not every construct can be typed. (type = object/data type, not typing on keyboard :) -- steve at adam.com.au