itertools candidate: warehouse()

[Peter Otten]

Robert Brewer wrote:

> def warehouse(stock, factory=None):

[snip documentation]

>     if not hasattr(stock, 'next'):
>         stock = iter(stock)

Note that unconditional use of iter() is usually harmless:

>>> it = iter("abc")
>>> it is iter(it)
True
   
>     def pull():
>         """An inner generator from itertools.warehouse()."""
>         for item in stock:
>             yield item
>         
>         if factory is None:
>             try:
>                 local_factory = item.__class__
>             except NameError:
>                 raise ValueError("Empty sequence and no factory
> supplied.")
>         else:
>             local_factory = factory
>         
>         while True:
>             yield local_factory()
>     
>     return pull(), stock
> 
> 
> What do you all think? I've been using a class-based variant of this in
> production code (business app) for six months now, and have found it
> extremely helpful. I saw a pointer to itertools today and figured a
> function-based version might be nice to include in that module someday.

Most of the building blocks for the warehouse() are already there, but you
didn't use them, oddly enough.
So here comes a variant written in terms of current (2.4) itertools:

>>> from itertools import *
>>> def peek(iterable):
...     a, b = tee(iterable)
...     try:
...             return a, b.next()
...     except StopIteration:
...             raise ValueError("cannot peek into an empty iterable")
...
>>> iterable = iter("abc")
>>> iterable, first = peek(iterable)
>>> factory = first.__class__
>>> data = range(10)
>>> for a, b in izip(data, chain(iterable, starmap(factory, repeat(())))):
...     print a, repr(b)
...
0 'a'
1 'b'
2 'c'
3 ''
4 ''
5 ''
6 ''
7 ''
8 ''
9 ''

Ok, I see I'm suffering from a functional overdose :-)
I would have suggested chain(iterable, starmap(factory, repeat(()))) for
inclusion in the collection of recipes in the documentation, but upon
checking the development docs I see that Raymond Hettinger has already been
there, done that with the starmap() part.

So now you can do

chain(iterable, repeatfunc(factory))

after putting a copy of these recipes into your site-packages. Why aren't
they already there, btw?

The only extra your warehouse() has to offer is the (last) item's class as
the default factory for the padding items. I don't think that is needed
often enough to warrant the inclusion in the itertools.

Peter