Scanning a file

[Alex Martelli]

Bengt Richter <bokr at oz.net> wrote:
   ...
> >>>    while block:
> >>>        block = block[-overlap:] + f.read(blocksize-overlap)
> >>>        if block: yield block
   ...
> I was thinking this was an example a la Alex's previous discussion
> of interviewee code challenges ;-)
> 
> What struck me was
> 
>  >>> gen = byblocks(StringIO.StringIO('no'),1024,len('end?')-1)
>  >>> [gen.next() for i in xrange(10)]
>  ['no', 'no', 'no', 'no', 'no', 'no', 'no', 'no', 'no', 'no']

Heh, OK, I should get back into the habit of adding a "warning: untested
code" when I post code (particularly when it's late and I'm
jetlagged;-).  The code I posted will never exit, since block always
keeps the last overlap bytes; it needs to be changed into something like
(warning -- untested code!-)

if overlap>0:
  while True:
    next = f.read(blocksize-overlap)
    if not next: break
    block = block[-overlap:] + next
    yield block
else:
  while True:
    next = f.read(blocksize)
    if not next: break
    yield next

(the if/else is needed to handle requests for overlaps <= 0, if desired;
I think it's clearer to split the cases rather than to test inside the
loop's body).


Alex