itertools candidate: warehouse()

[Raymond Hettinger]

"Robert Brewer" <fumanchu at amor.org> wrote ...

def warehouse(stock, factory=None):
    """warehouse(stock, factory=None) -> iavailable, iremainder.
Iterate over stock, yielding each value. Once the 'stock' sequence is
exhausted, the factory function (or any callable, such as a class) is
called to produce a new valid object upon each subsequent call to
next().
    
 If factory is None, the class of the first item in the sequence is
used as a constructor. If the factory function is not a bound method, it
does not receive any arguments, so if your class has mandatory arguments
to __init__, wrap the class in a function which can supply those.
 . . .
What do you all think? I've been using a class-based variant of this in
production code (business app) for six months now, and have found it
extremely helpful. I saw a pointer to itertools today and figured a
function-based version might be nice to include in that module someday.

------------------------------------------------------------
Raymond's reply ...

Here are some random thoughts based on a first read through:

* The basic idea of iterating stock and then calling a factory when the
stock is depleted is already expressible in terms of itertools.  For
example, if random.random() is the factory:

   iavailable = chain(stock, starmap(random.random, repeat(())))

* The second iterator, iremainder, ventures well beyond the basic idea
of a warehouse/factory chain.  It pushes the learning curve up steeply
and makes the function less generic (i.e. plenty of situations that
could use the warehouse/factory chain have no need for the separate
iremainder iterator).

* Making the factory argument optional is, IMO, also trying to do too
much.  Using the first stock item to identify a type and then calling
the type is probably more appropriate for prototype based OO (i.e. with
objects designed for cloning operations).  Also, the number of potential
uses is limited because knowing the type does not imply that you know
the signature for the constructor or that you can infer argument values
for constructors that take arguments. 

* This function has not seen real world use outside your organization.
Posting it as an ASPN recipe might give it a chance to be kicked around
a bit and perhaps re-emerge in a more mature form.

* Most of the itertools and their kin only try to encapsulate a tiny bit
of logic (enumerate and takewhile for example).  Even the most complex
tool, groupby(), is built around an atomic idea (roughly akin to the
uniq filter in Unix uniq).  The bite sized approach enables itertools to
simplify code without obscuring the program logic.  IMO, the proposed
warehouse() function does not share that advantage (though it is
possible that repetition may allow ones mind to eventually condense the
logic into a managable unit of thought).


Raymond Hettinger