speedy Python strings?

[Francis Avila]

Stuart D. Gathman wrote in message ...
>#         text = struct.unpack("L", self.buffer[:4])
>#         self.buffer = self.buffer[4:]
>   pos = self.pos
>   text = struct.unpack("L", self.buffer[pos:pos+4])
>   self.pos = pos + 4


In this vein, I would also recommend looking at the array module.  You
didn't describe the data structure, but if each record is simply a list of
4-byte integers, you could convert the whole record to ints at once like so:

record = array.array('L', stringorlist)
Then, perform your loops on record, which will be a list-like object
supporting item insertion and deletion, and conversion to bytestrings and
other Python base types.

Or, you could simply use array.array() to implement your buffer more
efficiently than continually calling struct.  Simply buffer the converted
data a chunk at a time, using array.

However, I don't think buffering helps much in this case.  File operations
are already buffered by Python, and most (all?) OSes buffer reads
themselves, too.  Reading a file is almost always very fast.  I would
concentrate more on presenting a useful abstraction for your data, rather
than worrying about what is essentially an optimization problem.
--
Francis Avila