chunking a long string?

[Peter Otten]

Roy Smith wrote:

> I have a long string (several Mbytes).  I want to iterate over it in
> manageable chunks (say, 1 kbyte each).  For (a small) example, if I
> started with "this is a very long string", and I wanted 10 character
> chunks, I should get:
> 
> "this is a "
> "very long "
> "string"
> 
> This seems like something itertools would do, but I don't see anything. 
> Is there something, or do I just need to loop and slice (and worry about
> getting all the edge conditions right) myself?

(x)range() can take care of the edges:

>>> s = "this is a very long string"
>>> def chunks(s, size):
...     for start in xrange(0, len(s), size):
...             yield s[start:start+size]
... 
>>> list(chunks(s, 10))
['this is a ', 'very long ', 'string']
>>> list(chunks(s, 5))
['this ', 'is a ', 'very ', 'long ', 'strin', 'g']
>>> list(chunks(s, 100))
['this is a very long string']

Or you use StringIO:

>>> from functools import partial
>>> from StringIO import StringIO
>>> list(iter(partial(StringIO(s).read, 5), ""))
['this ', 'is a ', 'very ', 'long ', 'strin', 'g']

And no, this need not be a one-liner ;)