[Python-Dev] Re: Feature bloat in Python (was some PEP thing!)

[Skip Montanaro]

    Fredrik> I cannot really comment on the list comprehension syntax since
    Fredrik> the PEP isn't there yet, but if I remember the syntax
    Fredrik> correctly, maybe "list confusions" is a better name than "list
    Fredrik> comprehensions".  I surely don't understand them...

Let me take a crack at explaining list comprehensions one more time.  There
is a patch available to play with, btw:

    https://sourceforge.net/patch/?func=detailpatch&patch_id=100654&group_id=5470

As current implemented (which is probably not precisely what people have
been talking about), a list comprehension is simply a create a new list from
one or more other lists to new using a series of constraint expressions.

Working through some simple examples from the modified test_grammar.py:

    * multiplying a list by a constant...

    >>> nums = [1,2,3,4,5]
    >>> print [3*x for x in nums]
    [3, 6, 9, 12, 15]

    You can read this as, "for all values 3x such that x is in nums".

    * selecting a subset of a list...

    >>> print [x for x in nums if x > 2]
    [3, 4, 5]

    You can read this as, "for all values x such that x is in nums and x is
    greater than 2".

    * combining two lists...

    >>> strs = ["Apple", "Banana", "Coconut"]
    >>> print [i, s for i in nums for s in strs]
    [(1, 'Apple'), (1, 'Banana'), (1, 'Coconut'), (2, 'Apple'), (2, 'Banana'), (2, 'Coconut'), (3, 'Apple'), (3, 'Banana'), (3, 'Coconut'), (4, 'Apple'), (4, 'Banana'), (4, 'Coconut'), (5, 'Apple'), (5, 'Banana'), (5, 'Coconut')]

    You can read this as, "for all pairs (i,s) such that i is in nums and s
    is in strs".

    * nesting list comprehensions ...

    >>> print [i, s for i in nums for s in [f for f in strs if "n" in f]]
    [(1, 'Banana'), (1, 'Coconut'), (2, 'Banana'), (2, 'Coconut'), (3, 'Banana'), (3, 'Coconut'), (4, 'Banana'), (4, 'Coconut'), (5, 'Banana'), (5, 'Coconut')]

    You can read this as, "for all pairs (i,s) such that is is in nums as s
    is in the subset of nums that contains the letter 'n'".

I don't think it's so different than the notation mathematicians use to
describe and build sets.  We don't have the luxury of convenient glyphs that
mean "there exists" and "for all", however.

As has been pointed out, the syntax is constrained by the demands of the
current parser and preexisting expression syntax.  For instance, we can't
currently put "and" between the different clauses to clearly indicate that
we are not nesting for statements without creating ambiguities.  I don't
know, perhaps that's simply an argument for stronger parsing technology for
Python.  If so, and a stronger parser would allow a more readable
implementation of list comprehensions, I'd be in favor of delaying their
introduction until such a parser was in place.  I'm not the person to make
the decisions about a new parser.  Would (for example) a yacc/bison and
lex/flex based parser be completely out of the question in the 2.1 to
2.2 timeframe?  Would it allow a more expressive grammar?  What portability
issues would be involved?

Skip