Itertools wishlists

[Ville Vainio]

>>>>> "Raymond" == Raymond Hettinger <vze4rx4y at verizon.net> writes:

    Raymond> Each one of the options listed is a reason that flatten()
    Raymond> shouldn't be an itertool.  It fails tests of obviousness,
    Raymond> learnability, complexity of implementation, and
    Raymond> simplicity of API.  The options also suggest that the
    Raymond> abstraction is not as basic or universal as we would
    Raymond> hope.

A simpler API:

def flatten(sequence, atomic_test = lambda o: isinstance(o,basestring)):
  """ don't recurse into iterables if atomic_test -> True """

I believe speaking of the "levels" of flattening is contorted here.

    Raymond> Perhaps "require" was the wrong word.  The issue is that
    Raymond> appear to be very few real situations where flatten()
    Raymond> would be the tool of choice.

Suppose that I get a very complex data structure involving lists of
tuples of tuples [....] of strings. I just want to quickly search the
sequence for valid file names, without going through elaborate
unpacking. Then I just do

files = (f fof f in flatten(monster_data_struct) if os.path.isfile(str(f)))

Yep, this is a real use case (ipython + some of my own data munging
tools).

    Raymond> Generalizing the two results, it may be fair to say that
    Raymond> the desire to flatten is a code smell indicating that
    Raymond> structure is being unnecessarily destroyed or that
    Raymond> earlier processing introduced unwanted structure.  Let
    Raymond> the data guide the programming.

You are looking the problem from a specific mindset, that of writing
good clean pythonic code. flatten is for situations when you need an
implementation 20 seconds ago (where someone might have recommended
perl in the past, and which is a perfectly valid niche for Python as
well).

It's not a matter of life & death for me, obviously (it's in my own
stdlib). I still can't see how its existence would make rest of
itertools magically harder to learn. When I come up with a problem
where I imagine itertools might come in handy, I check the docs to see
whether there is anything appropriate for the problem. I don't
memorize all the functions, just the fact that such functions
exist.

Also, the following itertool functions are not very useful anymore,
with the advent of genexps:

ifilter(pred, seq) --> elements of seq where pred(elem) is True
ifilterfalse(pred, seq) --> elements of seq where pred(elem) is False
imap(fun, p, q, ...) --> fun(p0, q0), fun(p1, q1), ...
starmap(fun, seq) --> fun(*seq[0]), fun(*seq[1]), ...

I don't believe a genuinely useful 'flatten' would increase the
cognitive load any more than these.

-- 
Ville Vainio   http://tinyurl.com/2prnb