Iterating across a filtered list

[Paul Rubin]

Bruno Desthuilliers <bdesth.quelquechose at free.quelquepart.fr> writes:
> I don't know if I qualify as a Python traditionalist, but I'm using
> Python since the 1.5.2 days, and I usually favor list comps or
> generator expressions over old-style loops when it comes to this kind
> of operations.

I like genexps when they're nested inside other expressions so they're
consumed as part of the evaluation of the outer expression.  They're a
bit scary when the genexp-created iterator is saved in a variable.

Listcomps are different, they allocate storage for the entire list, so
they're just syntax sugar for a loop.  They have an annoying
misfeature of their

> Python has had functions as first class objects and
> (quite-limited-but) anonymous functions, map(), filter() and reduce()
> as builtin funcs at least since 1.5.2 (quite some years ago).

True, though no iterators so you couldn't easily use those functions
on lazily-evaluated streams like you can now.

> >  Iterators like that are a new Python feature
> List comps are not that new (2.0 or 2.1 ?):
> print "\n".join([contact for name, contact in contacts.items() \
>                   if search.match(name)])

Well you could do it that way but it allocates the entire filtered
list in memory.  In this example "\n".join() also builds up a string
in memory, but you could do something different, like run the sequence
through another filter or print out one element at a time, in which
case lazy evaluation can be important (imagine that contacts.iteritems
chugs through a billion row table in an SQL database).

> > It's usually safest to create and consume them in the same
> > place, e.g. creating some sequence and passing it through map, filter, etc.
> Safest ? Why so ?

Just that things can get confusing if you're consuming the iterator in
more than one place.  It can get to be like those old languages where
you had to do your own storage management ;-).