python newbie - slicing a big memory chunk without GC penalties

[Jp Calderone]

On Sun, Feb 02, 2003 at 06:06:59PM +0000, Giovanni Bajo wrote:
> Hello,
> 
> Sorry if the question is trivial, but I am a newbie with Python. I have a
> file read into memory within a 'sequence' (or whatever is returned by
> file.read()), and I need to process it 512 bytes a time. Now, I was doing
> something like:
> 
> for i in range(0, len(buf)/512):
>     Process(buf[i*512 : (i+1)*512])
> 

  Not that it's relevant, but...

  for i in range(0, len(buf), 512):
    Process(buf[i:i + 512])

  The time it takes to slice one of the built-in sequences depends
*greatly* on the sequence type.  Slicing strings is an order of magnitude
faster than slicing lists, and slicing tuples is almost as fast.  What kind
of type are you working with?  Are you planning to change the data?  If it's
a list, maybe switching to a tuple would be enough of a speedup.

> But it seems like a lot of time is wasted in the sequence slicing (before I
> was processing everything in a shot, and it was much faster - and Process is
> O(n) so it should not really matter that much). I tried also other
> approacches like:
> 
> while len(buf):
>     Process(buf[0:512])
>     buf = buf[512:]
> 

  Yep, that's nasty.  ;)

  Jp

-- 
A sad spectacle.  If they be inhabited, what a scope for misery 
and folly.  If they be not inhabited, what a waste of space.
                -- Thomas Carlyle, looking at the stars
-- 
 up 48 days, 21:50, 4 users, load average: 0.27, 0.16, 0.06
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20030202/3584d00c/attachment.sig>