[Python-ideas] Batching/grouping function for itertools

[Steven D'Aprano]

On Sun, Dec 08, 2013 at 05:02:05PM +1000, Nick Coghlan wrote:

> The windowing problem is too ill-defined - there are enough degrees of
> freedom that any API flexible enough to cover them all is harder to
> learn than just building out your own version that works the way you
> want it to, and a more restrictive API that *doesn't* cover all the
> variants introduces a sharp discontinuity between the "blessed"
> variant and the alternatives.

Playing Devil's Advocate here, I wonder if that is true though. It seems 
to me that there are two basic windowing variants: sliding windows, and 
discrete windows. That is, given a sequence [a, b, c, d, e, f, g, h, i] 
and a window size of 3, the two obvious, common results are:

# sliding window
(a,b,c), (b,c,d), (c,d,e), (d,e,f), (e,f,g), (f,g,h), (g,h,i)

# discrete windows
(a,b,c), (d,e,f), (g,h,i)


Surely anything else is exotic enough that there is no question about 
leaving it up to the individual programmer.

In the second case, there is a question about what to do with sequences 
that are not a multiple of the window size. Similar to zip(), there are 
two things one might do:

- pad with some given object;
- raise an exception

If you want to just ignore extra items, just catch the exception and 
continue. So that's a maximum of three window functions:

sliding(iterable, window_size)
discrete(iterable, window_size, pad=None)
strict_discrete(iterable, window_size)

or just two, if you combine discrete and strict_discrete:

discrete(iterable, window_size [, pad])
# raise if pad is not given


What other varieties are there? Surely none that are common. Once, for a 
lark, I tried to come up with one that was fully general -- as well as a 
window size, you could specify how far to advance the window each step. 
The sliding variety would advance by 1 each step, the discrete variety 
would advance by the window size. But I never found any reason to use it 
with any other step sizes. Surely anything else is more useful in theory 
than in practice.

(That's three times I've said something is "surely" true, always a sign 
my argument is weak *grin*)

Given that this windowing problem keeps coming up, there's no doubt in 
my mind that it is a useful, if not fundamental, iterator operation. 
Ruby's Enumerable module includes both:

http://ruby-doc.org/core-2.0.0/Enumerable.html

each_cons is what I've been calling a sliding window, and each_slice is 
what I've been calling discrete chunks.



-- 
Steven