Iterate through list two items at a time

[Wade Leftwich]

Peter Otten wrote:
> Wade Leftwich wrote:
>
> > from itertools import groupby
> >
> > def chunk(it, n=0):
> >     if n == 0:
> >         return iter([it])
> >     def groupfun((x,y)):
> >         return int(x/n)
> >     grouped = groupby(enumerate(it), groupfun)
> >     counted = (y for (x,y) in grouped)
> >     return ((z for (y,z) in x) for x in counted)
> >
> >>>> [list(x) for x in chunk(range(10), 3)]
> > [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
> >
> >>>> [x for x in chunk(range(10), 3)]
> > [<generator object at 0xb7a34e4c>,
> >  <generator object at 0xb7a34dac>,
> >  <generator object at 0xb7a34d2c>,
> >  <generator object at 0xb7a34d6c>]
>
> Note that all but the last of these generators are useless:
>
> >>> chunks = [x for x in chunk(range(10), 3)]
> >>> [list(x) for x in chunks]
> [[], [], [], [9]] # did you expect that?
> In [48]: chunkgen = chunk(range(10), 3)

In [49]: for x in chunkgen:
   ....:     print list(x)
   ....:
   ....:
[0, 1, 2]
[3, 4, 5]
[6, 7, 8]
[9]

> Peter

That's an interesting gotcha that I've never run into when using this
function, because in practice I never put the generator returned by
chunk() inside a list comprehension.

In [51]: chunkgen = chunk(range(10), 3)

In [52]: [list(x) for x in chunkgen]
Out[52]: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]

But, as you pointed out --

In [57]: chunkgen = chunk(range(10), 3)

In [58]: chunks = list(chunkgen)

In [59]: [list(x) for x in chunks]
Out[59]: [[], [], [], [9]]

So apparently when we list(chunkgen), we are exhausting the generators
that are its members. And if we do it again, we get rid of the [9] --

In [60]: [list(x) for x in chunks]
Out[60]: [[], [], [], []]

I'll admit that chunk() is needlessly tricky for most purposes. I wrote
it to accept a really lengthy, possibly unbounded, iterator and write
out 10,000-line files from it, and also to play with Python's new
functional capabilities.

-- Wade