From H.Zahiri at curtin.edu.au Sun Jan 11 14:08:54 2009 From: H.Zahiri at curtin.edu.au (Hani Zahiri) Date: Sun, 11 Jan 2009 22:08:54 +0900 Subject: [melbourne-pug] Problem with reading binary file (different result between MATLAB and Python) Message-ID: <82200558F6DE2C479D381D3000D1551C04DC9014@EXMSK1.staff.ad.curtin.edu.au> Hi folks, I am trying to translate one of my MATLAB scripts to Python and I am experiencing a strange problem (at least to me!) and I am desperetly looking for help. The binary file is a raw binary containing header information (first 720 bytes) following by radar data. For better illustration and using python basic functions, first 800 bytes of the file is look like this: >>> fid = open("file_name","rb") >>> fid.read(800) '\x00\x00\x00\x012\xc0\x12\x12\x00\x00\x02\xd0A CEOS-SAR-CCT A A 1.00 1AL1 PSRBIMOP FSEQ 1 4FTYP 5 4FLGT 9 4 18432 88220 32 2 8 1 18432 0 10976 0 0 0BSQ 1 1 412 87808 0 13 4PB 49 2PB 45 4PB 21 4PB 29 4PB 97 4PB COMPLEX*8 C*8 0 0 \x00\x00\x00\x022\n\x12\x14\x00\x01X\x9c\x00\x00\x00\x01\x00\x00\x00\x01 \x00\x00\x00\x00\x00\x00*\xe0\x00\x00\x00\x00\x00\x00\x00\x00\ ... x00\x00\x07\xd6\x00\x00\x00\x8d\x02\xdd\xfe\x95\x00\x01\x00\x00\x00\x00\ x00\x00\x00\x1c\xae\x93\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ix ... \x00\x00\x00\x00I}.\xd0' And now the problem is: If I read the file in MATLAB, let say to find out length of header part, I will get the correct answer: EDU>> fid=fopen('file_name','r','b'); EDU>> fseek(fid,8,'bof'); EDU>> fread(fid,1,'uint32') ans = 720 However if I read this in python I am keep getting this wrong: >>> fid.seek(8) >>> scipy.fromfile(fid,'uint32',1) array([3489792000], dtype=uint32) I have almost tried every Scipy and Numpy classes with no result. I need a quick answer to this and I appreciate if anybody can help me with this problem. Cheers, Hani -------------- next part -------------- An HTML attachment was scrubbed... URL: From H.Zahiri at curtin.edu.au Sun Jan 11 13:53:31 2009 From: H.Zahiri at curtin.edu.au (Hani Zahiri) Date: Sun, 11 Jan 2009 21:53:31 +0900 Subject: [melbourne-pug] Problem with reading binary file (different result between MATLAB and Python) Message-ID: <82200558F6DE2C479D381D3000D1551C04DC9011@EXMSK1.staff.ad.curtin.edu.au> Hi folks, I am trying to translate one of my MATLAB scripts to Python and I am experiencing a strange problem (at least to me!) and I am desperetly looking for help. The binary file is a raw binary containing header information (first 720 bytes) following by radar data. For better illustration and using python basic functions, first 800 bytes of the file is look like this: >>> fid = open("file_name","rb") >>> fid.read(800) '\x00\x00\x00\x012\xc0\x12\x12\x00\x00\x02\xd0A CEOS-SAR-CCT A A 1.00 1AL1 PSRBIMOP FSEQ 1 4FTYP 5 4FLGT 9 4 18432 88220 32 2 8 1 18432 0 10976 0 0 0BSQ 1 1 412 87808 0 13 4PB 49 2PB 45 4PB 21 4PB 29 4PB 97 4PB COMPLEX*8 C*8 0 0 \x00\x00\x00\x022\n\x12\x14\x00\x01X\x9c\x00\x00\x00\x01\x00\x00\x00\x01 \x00\x00\x00\x00\x00\x00*\xe0\x00\x00\x00\x00\x00\x00\x00\x00\ ... x00\x00\x07\xd6\x00\x00\x00\x8d\x02\xdd\xfe\x95\x00\x01\x00\x00\x00\x00\ x00\x00\x00\x1c\xae\x93\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ix ... \x00\x00\x00\x00I}.\xd0' And now the problem is: If I read the file in MATLAB, let say to find out length of header part, I will get the correct answer: EDU>> fid=fopen('file_name','r','b'); EDU>> fseek(fid,8,'bof'); EDU>> fread(fid,1,'uint32') ans = 720 However if I read this in python I am keep getting this wrong: >>> fid.seek(8) >>> scipy.fromfile(fid,'uint32',1) array([3489792000], dtype=uint32) I have almost tried every Scipy and Numpy classes with no result. I need a quick answer to this and I appreciate if anybody can help me with this problem. Cheers, Hani -------------- next part -------------- An HTML attachment was scrubbed... URL: From sjmachin at lexicon.net Sun Jan 11 21:31:43 2009 From: sjmachin at lexicon.net (John Machin) Date: Mon, 12 Jan 2009 07:31:43 +1100 Subject: [melbourne-pug] Problem with reading binary file (different result between MATLAB and Python) In-Reply-To: <82200558F6DE2C479D381D3000D1551C04DC9014@EXMSK1.staff.ad.curtin.edu.au> References: <82200558F6DE2C479D381D3000D1551C04DC9014@EXMSK1.staff.ad.curtin.edu.au> Message-ID: <496A572F.9010307@lexicon.net> On 12/01/2009 12:08 AM, Hani Zahiri wrote: > Hi folks, > I am trying to translate one of my MATLAB scripts to Python and I am > experiencing a strange problem (at least to me!) and I am desperetly > looking for help. The binary file is a raw binary containing header > information (first 720 bytes) following by radar data. For better > illustration and using python basic functions, first 800 bytes of the > file is look like this: > >>> fid = open(?file_name?,?rb?) > > >>> fid.read(800) > > '\x00\x00\x00\x012\xc0\x12\x12\x00\x00\x02\xd0A CEOS-SAR-CCT A A [snip 700+ bytes of irrelevant stuff] > And now the problem is: > > If I read the file in MATLAB, let say to find out length of header part, > I will get the correct answer: > EDU>> fid=fopen('file_name','r','b'); > EDU>> fseek(fid,8,'bof'); > EDU>> fread(fid,1,'uint32') > ans = > 720 > However if I read this in python I am keep getting this wrong: > >>> fid.seek(8) > >>> scipy.fromfile(fid,'uint32',1) > array([3489792000], dtype=uint32) > > I have almost tried every Scipy and Numpy classes with no result. I need > a quick answer to this and I appreciate if anybody can help me with this > problem. Hello, Hani, Have you asked anywhere else? Flattered by the implication that melbourne-pug is the fount of all knowledge, somewhat dismayed by the thought that we're expected [in some sense] to be thinking Python at midnight on a Sunday :-) For future reference, news:comp.lang.python covers all timezones. Not being a numpy/scipy person, I'd use struct.unpack: | >>> hex(720) | '0x2d0' | >>> guff = '\x00\x00\x00\x012\xc0\x12\x12\x00\x00\x02\xd0A CEOS-SAR-CCT' | >>> guff[8:12] | '\x00\x00\x02\xd0' | >>> import struct | >>> struct.unpack('>I', guff[8:12]) | (720,) | >>> struct.unpack('>> Looks like the problem is inability to distinguish the big end from the little end :-) I'd suggest asking on the numpy/scipy mailing-list -- I'd be very surprised if there wasn't already a way of reading MATLAB files in Python at a higher level than using seek/fromfile/etc. HTH, John From richardjones at optushome.com.au Fri Jan 23 23:56:45 2009 From: richardjones at optushome.com.au (Richard Jones) Date: Sat, 24 Jan 2009 09:56:45 +1100 Subject: [melbourne-pug] Bids open to run OSDC 2009 Message-ID: <200901240956.46259.richardjones@optushome.com.au> The Open Source Developers' Conference has been run successfully for the last 5 years, drawing bigger crowds every year. For the first three of those years (2004-2006) it ran in Melbourne before moving to Brisbane (2007) and Sydney (2008). Each year it has had at least three full streams of content running concurrently, catered tea breaks and lunches, a formal sit-down conference dinner on the first night, and free speaker entry. Some years have also included one or two days of tutorial content. The Open Source Developers' Conference provides an important opportunity for developers of open source applications and developers who use open source tools to share knowledge and teach each other about the changes and new ideas that have come up in the last year or so. Our core focus is on writing code, and the tools that make that easier. The Open Source Developers' Club exec are seeking applications from interested parties to host the 2009 Open Source Developers' Conference. Location is not an issue, so long as you are in the oceanic region. If you are interested, or know someone that is, please complete the application form at http://tinyurl.com/osdc2009-application before 1st March 2009. The submission email address is included at the top of the form. We do have some model rules ( http://www.osdcon.org/wiki/Model_Rules ) and a lot of ideas to share. Having said that, we're willing to be as active or quiet as you need us to be. Traditionally the conference has been run in either the last week of November or the first week in December, if you want to deviate a lot from these dates you will need to talk to us first. You will need at group of 6-8 dedicated people who are able to give up an average of 3 hours a week from the acceptance date through to the conference itself. You will be responsible for: - finding and booking the venue(s) - managing your finances (there will be some seed capital) - advertising your call for papers to as many open source programming groups you know (we're happy to share our list with you too) - creating the program (preferably ensuring that no one language or concept unduly dominates) - identifying, deciding upon and inviting keynote speakers - organising all the social events - organising the catering - seeking and arranging sponsorships - advertising the conference - finding volunteers and running an awesome conference In return you will get: - help from us when asked - the satisfaction of arranging an awesome conference - fame for running the best OSDC yet - a chance to improve your profile in the open source world - kudos from the exec committee, speakers and attendees As this is such a big undertaking, we want you to know that we'll support you every step of the way. Should you need more help, we're here to offer advice or step in and do things if we can. Although OSDC Inc isn't flush with cash, there are funds to cover deposits before the money comes in. If for unforeseen difficulties the conference makes a loss, you won't be out of pocket; although we do ask that you keep us apprised of the financial situation on a regular basis and warn us if there may be a problem. We look forward to seeing your application! The Open Source Developers' Club Exec