list comprehensions whats happening here

[Carlos Ribeiro]

First of all, I agree with Steve's conclusions. I never said that my 
"dont't work" example should work, because it is clearly not a list 
comprehension. And yes, the syntax is clearly legitimate, if you remember 
that the comma makes a one-element tuple with range() list, and is not part 
of the "for" clause. My only (and small) concern is that the result of the 
construct below is not immediately obvious:

 >>> [[i,j] for i in range(3), for j in range(3)]
[[[0, 1, 2], 0], [[0, 1, 2], 1], [[0, 1, 2], 2]]

However, if we write...

 >>> [[i,j] for i in (range(3),) for j in range(3)]
[[[0, 1, 2], 0], [[0, 1, 2], 1], [[0, 1, 2], 2]]

the resulting code is much cleaner. As Emile Sebille pointed out, one way 
to solve the problem would be to demand that the single element tuple 
should be encapsulated in parenthesis. However, that would be too much, and 
could also break older code if done extensively.

As it is, I'm already convinced that the syntax is ok. Maybe a note on the 
documentation could be made as a warning for this particular idiom; maybe a 
general recommendation to always write one-element tuple inside 
parenthesis, to avoid confusion like the one above.

For example:

"""
One particular useful list comprehension idiom is as follows:

[[i,j] for i in (a, ) for i in b]

where "b" is a sequence, and "a" may be anything. This idiom will
prepend all elements in "b" with the value of "a". It is related
to both map() and zip(). However, map() will prepend "a" only
to the first element of "b", prepending None to the rest of the
sequence; and zip() always stops on the shortest list, which will
makes it return a list containing only the first element of b,
prepended by a.

For example,

 >>> [[i,j] for i in (0,) for j in ('a', 'b', 'c')]
[[0, 'a'], [0, 'b'], [0, 'c']]

prepends 0 to all members of the list;

 >>> map(None, (0,), ('a', 'b', 'c'))
[(0, 'a'), (None, 'b'), (None, 'c')]

prepends only the first element;

 >>> zip((0,), ('a', 'b', 'c'))
[(0, 'a')]

prepends and returns only the first element.
"""

Maybe someone that writes in english better than me could refine this, and 
make a recipe out of it. I also believe that the idiom above will turn out 
to be the fastest way to prepend all elements of a list with something, 
without using any extra space, or calling any user defined function.


Carlos Ribeiro


At 15:19 13/04/01 -0400, Steve Holden wrote:
>"Carlos Ribeiro" <cribeiro at mail.inet.com.br> wrote ...
> > At 07:06 13/04/01 -0700, Emile van Sebille wrote:
> > >I suspect the answer lies somewhere in the comma converting the parts of
>the
> > >list comprehension into a list itself, much as the comma is used in
>normal
> > >iteration idiom:
> > >
> > >for i in 1,2
> >
> > As other have pointed out, your're right, that does explain why it works
> > that way. But...
> >
> >  >>>[[i,j] [1,2,3], for j in 'abc',]
> > Traceback (  File "<interactive input>", line 1
> >      [[i,j] [1,2,3], for j in 'abc',]
> >
> > does not work, yet
> >
>Why should this work? A list comprehension substitutes one or more bound
>variables into a legitimate expression to generate the list members. But
>
>     [i,j] [1,2,3]
>
>is *not* a legitimate expression: the interpreter will complain because it
>thinks you are trying to subscript the list [i,j] with a tuple 1,2,3 -- list
>subscripts have to be integers.
>
> >  >>> [[i,j] for i in range(3), for j in range(3)]
> > [[[0, 1, 2], 0], [[0, 1, 2], 1], [[0, 1, 2], 2]]
> >
> > work.
> >
>Indeed. Let's take this a piece at a time... start by giving j a fixed value
>42.
>
>     [[i,j] for i in range(3),]
>
>is a legitimate expression: it is equivalent to
>
>     [[i,j] for i in (range(3), )]
>
>In other words, it produces a list of [i,j] lists, for i taking on every
>value of the tuple (range(3), ) - a single-element tuple, so the result from
>either of these expressions would be
>
>     [[[0, 1, 2], 42]]
>
>This is correct: we get a single element list whose only element is the list
>[i, j], where i was range(3) (the only value in the single-element tuple i
>was iterating across)  and j was 42.
>
>Let's look at your expression again, using a different range for j just so
>it's clearer which contributes to what:
>
>     [[i,j] for i in range(3), for j in range(6,9)]
>
>should give you a list containing three elements, since now j takes on not
>just a single value but three. Sure enough, the result is
>
>     [[[0, 1, 2], 6], [[0, 1, 2], 7], [[0, 1, 2], 8]]
>
>This is a list of three elements, one for each value of j. Each element of
>the list is a list of two elements, the first of which is the value of i
>(which is to say, range(3), or [0,1,2]) followed by a value of j (which is
>to say 6, then 7, then 8).
>
>And that's why
>
>[[i,j] for i in range(3), for j in range(3)]
>
>evaluates to
>
>[[[0, 1, 2], 0], [[0, 1, 2], 1], [[0, 1, 2], 2]]
>
>It's also why
>
>[[i,j] for i in range(3), for j in range(6,9),] # note second trailing comma
>
>has only one value - because both i and j are iterating over single-element
>tuples.
>
>[[[0, 1, 2], [6, 7, 8]]]
>
>
> > I am not advocating that something is wrong; I just want to point out that
> > some funny things are happening as a side effect of the grammar and/or
> > compiler. It's impossible to supply a list as part of the list
> > comprehension syntax, but you can fake it by writing a comma after the for
> > clause. That's just weird, and unexpected.
> >
>Nope, that's a tuple, not a list. Just so happened that it's a
>single-element tuple, whose only member is a list. If it's any help, I agree
>it's weird and unexpected, but you have to realise we're out on the edges
>here.
>
> > Question: this may turn out to be wrong, and it may be fixed in the
>future;
> > OTOH, it can be useful, and (in a rather obscure way) consistent with
> > Python syntax. What's the take on this? Let it be, or fix it, as a
> > undesirable side effect that don't belong to list comprehensions?
> >
>My own take is that everything is entirely consistent, and should be left
>alone. There's a good reason why the list comprehension syntax doesn't have
>commas in it!
>
>Hope this helps.
>
>regards
>  Steve
>
>
>
>--
>http://mail.python.org/mailman/listinfo/python-list