unintuitive for-loop behavior

[Rustom Mody]

On Saturday, October 1, 2016 at 11:35:31 PM UTC+5:30, Steve D'Aprano wrote:
> On Sun, 2 Oct 2016 03:57 am, Rustom Mody wrote:
> 
> > Hoo boy1
> > Thats some tour de force and makes my head spin
> 
> I certainly agree with the second part of your sentence.
> 
> 
> > Point can be made more simply with map
> > ie if we *define*
> > [exp for cv in l]
> > as
> > map(lambda cv: exp, l)
> > 
> > the problem vanishes
> 
> > 
> > Demo:
> > 
> > First a helper function for demoing:
> > 
> > def pam(fl,x):
> >     return map(lambda f: f(x), fl)
> > # pam is the complement to map; map runs one fnc on a list of args
> > # pam runs a list of funcs on one arg
> > 
> > Trying to make a list of functions that add one, two and three to their
> > arguments
> > 
> > fl = [lambda x: x + cv for cv in [1,2,3]]
> > 
> > Broken because of python's wrong LC semantics:
> >>>> pam(fl, 3)
> > [6, 6, 6]
> 
> Its not *broken*, its doing *exactly what you told it to do*. You said,
> define a function that takes a single argument x, and return x + cv. Then
> you delayed evaluating it until cv = 3, and passed the argument 3, so of
> course it returns 6. That's exactly what you told it to calculate.
> 
> You seem to have the concept that lambda should be magical, and just
> miraculously know how far back in time to look for the value of cv. And
> then when it doesn't, you're angry that Python is "broken". But why should
> it be magical? cv is just an ordinary variable, and like all variables,
> looking it up returns the value it has at the time you do the look-up, not
> some time in the past. Let's unroll the loop:
> 
> fl = []
> cv = 1
> def f(x): return x + cv
> fl.append(f)
> cv = 2
> def f(x): return x + cv
> fl.append(f)
> cv = 3
> def f(x): return x + cv
> fl.append(f)
> 
> pam(fl, 3)
> 
> Are you still surprised that it returns [6, 6, 6]?
> 
You are explaining the mechanism behind the bug. Thanks. The bug remains.
My new car goes in reverse when I put it in first gear but only on full-moon 
nights with the tank on reserve when the left light is blinking
The engineer explains the interesting software bug in the new PCB.
Interesting. But the bug remains

Likewise here:

[2 ** i for i in[1,2]] == [2**1, 2**2]

yet this fails 

[lambda x: x + i for i in [1,2]] == [lambda x:x+1, lambda x:x=2]

is a bug for anyone including the OP of this thread

> 
> 
> 
> > Transform the LC into a map with the rule above:
> > fl_good = map((lambda cv :lambda x: x+cv), [1,2,3])
> 
> 
> This is equivalent to something completely different, using a closure over
> cv, so of course it works:
> 
> def factory(cv):
>     def inner(x):
>         return x + cv
>     return inner
> 
> fl_good = []
> fl_good.append(factory(1))
> fl_good.append(factory(2))
> fl_good.append(factory(3))
> 
> 
> Each time you call factory(), you get a new scope, with its own independent
> variable cv. The inner function captures that environment (a closure),
> which includes that local variable cv. Each invocation of factory leads to
> an inner function that sees a different local variable which is independent
> of the others but happens to have the same name. Instead of three functions
> all looking up a single cv variable, you have three functions looking up
> three different cv variables.
> 
> This is essentially why closures exist.

Closures and lambdas are synonymous:
martin Fowler's article on lambda has this first sentence:

«a programming concept called Lambdas (also called Closures, Anonymous 
Functions or Blocks)»
http://martinfowler.com/bliki/Lambda.html

So if you like you can say the bug here is that python lambdas fail to close
variables properly.
Which is a double-bug because the blame for closure-failure is falling
onto the lambda whereas it is the comprehension semantics in terms of loops
rather than in terms of closures/cells as Jussi/Greg are trying to show, that
is at fault
> 
> 
> > Which is not very far from the standard workaround for this gotcha:
> >>>> fl_workaround = [lambda x, cv=cv: x+cv for cv in [1,2,3]]
> >>>> pam(fl_workaround, 3)
> > [4, 5, 6]
> >>>> 
> > 
> > Maybe we could say the workaround is the map definition uncurried
> > And then re-comprehension-ified
> 
> If your students think in terms of map, then fine, but I think it would
> confuse more people than it would help. Your mileage may vary.

You are as usual putting words in my mouth.
I did not talk of my students or that map is easier than comprehensions
I was taking the Haskell comprehension-semantics link (which BTW you posted!)
https://www.haskell.org/onlinereport/exps.html#sect3.11

Simplifying it to the simple case of only one for, no ifs
This allows the flatmap to become a map and the multiple cases to go
Thereby more succinctly highlighting the problem