Explanation of this Python language feature? [x for x in x for x in x] (to flatten a nested list)

[Rustom Mody]

On Sunday, March 23, 2014 8:16:28 AM UTC+5:30, Ian wrote:
> On Sat, Mar 22, 2014 at 6:32 PM, Rhodri James wrote:
> > wrote:
> >> Well almost...
> >> Except that the 'loop' I am talking of is one of
> >> def loop():
> >>      return [yield (lambda: x) for x in [1,2,3]]
> >> or
> >>      return (yield (lambda: x) for x in [1,2,3])
> >> or just plain ol
> >>      (lambda x:  for x in [1,2,3])
> >> IOW loop is an imperative construct, comprehensions are declarative
> > I'm sorry, you've made a logical leap too far here.  I understand loops
> > being imperative, but how are comprehensions declarative?  What do they
> > declare that the loop equivalent doesn't.

> I'm with Rustom on this point.  A list comprehension is a syntax for
> building a list by declaring a transformation from some other iterable
> object.  Forget comprehensions for a moment and think of literals.
> Would you not consider this to be declarative?

>     x = [1, 2, 3]

> A comprehension is syntactically similar to a literal, with just a
> different type of construction in mind.

Aha! Very elegantly put!

> Where I disagree is on the question of whether Python should therefore
> break its established closure rules for lambdas that are nested inside
> comprehensions versus functions that are not.

No... see below

> It breaks the
> equivalence between comprehensions and loops, and to my mind it
> introduces significant complexity for relatively little gain.

[I am not completely sure whether the following can be proved/is true]

1. One can change lambda's closure rules which would amount to
"significant complexity for relatively little gain"

2. One can change comprehension rules to not reassign to the 
running comprehension running varible but to rebind, using a recursive
function as the simulation of the comprehension rather than a for loop

3. 2 is semantically equivalent to 1
   - trivially for normal (ie non-lambda containing) expressions
   - and also for lambda containing expressions if your 
     default-argument trick is implemented by the python compiler
     [This is the claim I am not completely sure of and would love to hear 
     of/if counter examples]

Assuming its true:

One *semantically specifies* a comprehension with a recursive function, 
not a for loop

One *implements* a comprehension with the standard use of append
method inside a for as the expansion of a comprehension with the extra
caveat that interior lambdas are automatically wrapped inside a
default-argument binding for all outer comprehension variables.

A vanilla python programmer need not know anything about this any
more than a vanilla C programmer knows about
- strength reduction
- code hoisting
- loop unrolling
etc

that goes on inside an optimizing C compiler