Fast reading and unpacking of binary data (struct module)

[Gabriel Genellina]

En Tue, 21 Jul 2009 21:00:13 -0300, Daniel Platz  
<mail.to.daniel.platz at googlemail.com> escribió:

> I have an huge binary file from an external program. I want to read
> and process the data in this file in a reasonable time. It turns out
> that the reading of the data itself and the processing do not need
> most of the time. However, when using the read(bytes) method Python
> returns a string representing the binary information in hex. This
> string I have to "cast/translate" into a number (in my case a signed
> short). For this I am using the method struct.unpack from the struct
> module. This unpacking part of the program takes by far the most time.
> Is there a way to speed this up or to do it the unpacking more
> cleverly than with the struct module?

Try creating a Struct object with your format and use its unpack() method.
http://docs.python.org/library/struct.html#struct-objects

If your format consists of just integers, probably an array is more  
efficient:
http://docs.python.org/library/array.html#array.array.fromfile

-- 
Gabriel Genellina