map, filter, lambda, list comprehensions (was RE: parameter undefined in procedure)

[Tim Peters]

[Don Tuttle]
> Are there any plans to upgrade lambda, map, reduce and filter to become
> "major conveniences"?

P3K may introduce lexical closures (addressing one common gripe about
lambda) -- or may throw away lambda entirely.  Nobody knows yet.

As for the rest, from Guido in a recent Types-SIG msg:

    I'm not a functional programmer, and Python is not a functional
    programming language; and as far as I'm concerned it never will be one.

Of course, that's open to a wide variety of conflicting interpretations
<wink>.

> Or are there already better ways to accomplish the same things?

"better" is impossible to address without a metric.

Python classes and modules are very flexible.  Things done with closures in
other languages were intended to be done with explicit instance state in
Python.

reduce is almost always better (clearer & faster) done with an explicit
loop.

map and filter are often better done with an explicit loop.  map and filter
are most useful when you can pass a builtin function (e.g., str), or None (a
very special case for each -- see the docs).

> 1) I couldn't find any 50,000 ft overview of "list comprehensions".
> If one exists, would someone point me to it?

Hmm!  DejaNews doesn't seem able to find anything before 1999 on the subject
today, but the best discussion took place in the last half of '98.  I'll
attach one old intro msg of mine from 6 Aug 98.  It was later decided to
spell ":" as "if".

> 2) And without trying to re-kindle any passionate dialog, was there
> ever any general consenous reached about whether "list comprehensions"
> where likely to be included in Python?

The biggest attraction for Guido was in allowing to replace map and filter,
and many uses of lambda, with a much clearer ("more Pythonic") construct.
Quoth Guido:

    Better yet, it fixes the problems that Python's lambda has!

He means there that the proposed gimmick doesn't require lexical closures,
or artificial function calls.

But multi-argument uses of "map" require iterating over sequences in
parallel (lockstep, not nested), and Python's "for" can't express that
(directly) today.  Since "for" should act much the same whether in a loop or
a comprehension, it was decided that Python should first grow "parallel
iteration 'for'" syntax.  That was scheduled for 1.6, but looks like it's
getting delayed in the crunch to get Unicode support out the door.

That's as much consensus as there ever is <wink>.  In the meantime, Greg
Ewing released a patch that implements a large subset of the full list
comprehensions proposal (which you can find in DejaNews).

every-end-is-the-deep-end<0.5-wink>-ly y'rs  - tim



<BEGIN OLD MSG>

Subject: RE: Idea - alternative to lambda
Date: Thu, 6 Aug 1998 02:08:20 -0400

[Greg Ewing]
> It suddenly occurred to me this morning what
> Python wants to replace many of the uses of
> lambda in list-mapping operations:
>
> List comprehensions!
>
> def zip(L1, L2):
>   return [(x, y) for x in L1, y in L2]

[Paul Prescod]
> I don't understand the wider implications of your proposal. Can
> it do more than just tupleize multiple lists?

List comprehensions are a std feature of modern functional languages, to
which you can look for more examples.  You generally want an expression that
computes each element of the final list ("(x, y)" in the above), a
collection of iterators to bind names for use in the expression (the rest of
the above -- although I tend to read it as the cross product of L1 and L2
rather than a lockstep parallel iteration -- the syntax needs work), and an
optional filtering expression to further limit which elements get generated.

Some made-up examples:

    [x**2 for x in range(10)] == [0, 1, 4, 9, ..., 81]

    [f(x) for x in L] == map(f, L)

    [sqrt(x) for x in range(-1,2) : x >= 0] == [0., 1.]
    (reading ":" as "such that")

    [x for x in L] == list(L)

    [x for x in L : predicate(x)] == filter(predicate, L)

    [42 for x in range(N)] = [42] * max(N, 0)

    [(x, y**3) for x in range(3), for y in range(3): x < y] ==
        [(0, 1), (0, 8), (1, 8)] ==
        [(x, y**3) for x in range(3), for y in range(x+1, 3)]

    M = [(len(s), s) for s in stringList]
    M.sort()
    stringList = [x[1] for x in M]
        sorts a list of strings by length

    [f(pi) for f in (sin, cos, tan)] == [sin(pi), cos(pi), tan(pi)]

    [f(x, y) for x in L, for y in M(x): g(x, y)] ==
        X after
    <<save the current bindings for x and y>>
    X = []
    try:
        for x in L:
            for y in M(x):
                if g(x, y):
                    X.append(f(x, y))
    finally:
        <<restore the saved bindings for x and y>>

Note:  I avoided "the usual" transformation into nested lambdas, because
this much more efficient (in Python) flavor of implementation seems
*possible*:  given Guido's overwhelming fondness for the functional features
that snuck into Python while he was snoozing <wink>, efficiency may be the
only grounds on which it could be sold.

... [and other stuff] ...