itertools.flatten()? and copying generators/iterators.

[Francis Avila]

"Peter Otten" <__peter__ at web.de> wrote in message
news:bnqrni$mmn$05$1 at news.t-online.com...
> I think your flatten() is getting more complex fast, and you're reaching
the
> point where you should either use a version designed for the special
> purpose or pay the performance price and go with a simple test function
> like my keepstrings() somewhere above.

Actually, with your suggestion, it's gotten much simpler. See below.

> PS: The Martelli virus spreading fast: have you timed it recently?

I don't have time now, but I'll collect all the functions and time them all.
There are two groups though: those that can modify as they go (i.e. contain
custom type handlers), and those that don't, so I need two sets of test
cases, and to ensure that they all can at least pass the static tests.  And
there are a lot of functions.  But now I'm really curious to see their times
and the power/time tradeoff.

Anyway, final revision.  Much nicer (I took out the doctests cases for the
post. They're the same anyway, with one minor addition which I also added to
flatten_dictcond.)

--- Code ---

def defaulthandler(seq):
    """Return iterability of seq and new sequence.

    Iterability is either a bool (which asserts class-instance
    iterability), or a function which accepts a sequence and returns
    (instance-iterability, newseq).  The function asserts
    instance-iterability.

    Iterability as returned from default handler is *always* cached
    into do_descend, so for every call to this function, seq will have
    a unique type.

    I can't imagine that your own handler would *ever* need to modify
    seq before returning, unless you're doing some serious voodoo. You
    probably just need to add a type:function pair to do_descend.

    """
    # It is impossible to avoid an exception test, as far as I can
    # tell. Oh well.
    try:
        iter(seq)
    except TypeError:
        return False, seq
    else:
        return True, seq


def flatten_dictdef(iterable, do_descend=None,
defaulthandler=defaulthandler):
    """Yield leaves of iterable.

    Pretty much same semantics as flatten_dictcond.

    Function items in do_descend determine if iterable is instance-iterable.
        They return (iterability, newsequence), the iterability of the
        *NEW*SEQUENCE* (like normal).

    Boolean items in do_descend determine if iterable is
    class-iterable.

    Finally, defaulthandler determines any-iterability.
        defaulthandler returns (iterability, newsequence), the
        iterability of the *ORIGINAL*SEQUENCE*. (VERY important!)

    """
    ## Preload cache with atomic strings per default.
    if do_descend is None:
        do_descend = {type(''):False, type(u''):False}

    descend = do_descend.get(iterable.__class__, defaulthandler)

    # Of course, we end up *testing* in the reverse order of what the
    # docstring outlines.

    if descend is defaulthandler:
        # Oy, I was going to write this:

        # do_descend[iterable.__class__], iterable = defaulthandler(descend)

        # But I don't remember the order of evaluation, and I'm too
        # tired to look it up...

        isit, s = descend(iterable)
        descend = do_descend[iterable.__class__] = isit
        iterable = s

    elif callable(descend):
        descend, iterable = descend(iterable)
    #Else, bool constant.

    if not descend:
        yield iterable
    else:
        for elem in iterable:
            for subelem in flatten_dictdef(elem, do_descend,
defaulthandler):
                yield subelem
    # Ah, nice! This one I'm keeping.


--
Francis Avila