closures and dynamic binding

[Terry Reedy]

Paul Boddie wrote:
> On 29 Sep, 05:56, Terry Reedy <tjre... at udel.edu> wrote:
...
>> Dynamic: The namespace scope of 'n' in inner, how it is looked up, is
>> determined by where inner is called from. This is what you seemed to be
>> suggesting -- look up 'n' based on the scope it is *used* in.
...

> A somewhat separate issue is illustrated by the modification of x
> within f. Although for most statements, we would expect the value of x
> to evolve following from a top-to-bottom traversal of the code within
> a unit, function definition statements do not behave like, say, "for",
> "if" or "while" statements. Now although this should be obvious at the
> module global level, I feel that it can be easy to overlook within a
> function where one normally expects to find plain old control-flow
> constructs, expressions, assignments and so on. It is pertinent to
> note, with respect to the original inquiry, that lambda functions are
> subject to the same caveats,

Please: Python does not have 'lambda functions'.  Def statements and 
lambda expressions both define instances of the function class.  So this 
amounts to saying "functions are subject to the same caveats as functions."

 > and I believe that lexical scoping was introduced precisely to make
 > lambda functions less cumbersome to
> employ, eliminating the need to explicitly initialise them using the
> "identity" default parameters trick, but obviously introducing the
> consequences and potential misunderstandings described above.

As I meant when I wrote "Even without closures, the global scope of a 
function is the module it is defined in.", the concept of lexical 
scoping applies even to non-nested top-level functions.

Yes, closures for nested functions were introduced in part to replace 
the pseudo-default parameter trick (which does not require the confusion 
of using the same name in both local namespaces).  But this applies to 
both syntactical forms of function definition and had nothing to do in 
particular with easing the quite limited lambda form, which Guido does 
not like and once thought to delete in 3.0.

Closures were also introduced because default parameters are limited to 
capturing a value at the time of function definition instead of at 
function invocation.  Consider your example which I have moved down to here.

 > This is how I understand it, too. The confusing part involves the
 > definition of any inner function and how any "external" names are
 > bound or defined. As we've seen...
 >
 >   def f(x):
 >     def g():
 >       return x
 >     x += 1 # added for illustration
 >     return g
 >
 > ...it might look at first glance like the function known as g (within
 > f) should return the initial value of x (as known within f), since
 > that was the value x had when g was defined.

Only if one is confused about the difference between default parameter 
expressions, which are evaluated when the function is defined, and body 
expressions, which are evaluated when the function is called.  If you 
want g to use the 'initial' value that x has when g is defined, then say so.

def f(x):
   def g(y=x) # or x=x if you really prefer
     return y # or x, with the consequent possibility of confusion
   x += 1
   return g

If 'x' is fixed, this is a difference without effect.  If 'x' is not, as 
in this pair of examples, the difference is important.

The introduction of the 'nonlocal' keyword will make the latter case 
more common and hence the second reason more important.  Consider code like

def f(*args)
   x = [0]
   def g()
     ...
     x[0] += 1
     ...
   <code calling g some variable number of times>
   return x[0], ...

This has x fixedly bound to a particular list.  The default argument 
trick would have worked here.  In the future, this can and will be 
written more naturally as

def f(*args)
   x = 0
   def g()
     nonlocal x
     ...
     x += 1
     ...
   <code calling g some variable number of times>
   return x, ...

With x being rebound, the default argument trick would *not* work.  And, 
of course, lambda expressions are not in the picture.

Terry Jan Reedy