[Tutor] Generator next()

[Steven D'Aprano]

On Sun, Dec 29, 2013 at 01:57:31AM -0500, Keith Winston wrote:

> I don't really get the inner thing, I tried to look it up, but I don't
> think I found the right thing, just references to nested functions. I'd
> like to understand what I'm looking at better, but can't figure out what
> question to ask...

Let's start with a trivial example: a function inside another function.


def f(x):
    print("Inside the outer function x =", x)
    def g(y):  # a function nested inside another function
        print("Inside the inner function x =", x)
        print("Inside the inner function y =", y)
        return x + y
    return f(23)


And in use, in Python 3.3:

py> f(1000)
Inside the outer function x = 1000
Inside the inner function x = 1000
Inside the inner function y = 23
1023


The main benefit of writing the function this way is that it hides 
function g() from everything else. If I try to call it from outside of 
f(), I get an error:

py> g(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'g' is not defined


Note a few things:

- x is a local variable to function f(). f() is free to
  modify x without restriction.

- x is a non-local variable to the nested function g(). That 
  means it is neither local, nor global. By default, g()
  cannot modify x.

- y is a local variable to g(). This means that y is invisible 
  to f(), since it doesn't exist in the outer scope. In the 
  same way that code outside of f() cannot peer inside the 
  function and see its local variable x, so code outside of g()
  cannot peer inside of it to see its local variable y. That 
  includes f().


Now, let's crank it up a bit. In Python, functions are "first-class 
values" just like strings, numbers, lists and so forth. That means that 
you can return a function as the result. The usual name for this is a 
"factory function", or just factory, a function which creates a new 
function and returns it.

def adder_factory(n):
    def plus(arg):
        return arg + n
    return plus  # returns the function itself


If you call adder_factory(), it returns a function:

py> adder_factory(10)
<function adder_factory.<locals>.plus at 0xb7af6f5c>

What good is this? Watch carefully:


py> add_two = adder_factory(2)
py> add_three = adder_factory(3)
py> add_two(100)
102
py> add_three(100)
103


The factory lets you programmatically create functions on the fly. 
add_two() is a function which adds two to whatever argument it gets. 
add_three() is a function which adds three to whatever argument it gets. 
We can create an "add_whatever" function without knowing in advance what 
"whatever" is going to be:

py> from random import randint
py> add_whatever = adder_factory(randint(1, 10))
py> add_whatever(200)
205
py> add_whatever(300)
305


So now you see the main reason for nested functions in Python: they let 
use create a function where you don't quite know exactly what it will do 
until runtime. You know the general form of what it will do, but the 
precise details aren't specified until runtime.

There's another term you will come across from time to time: "closure". 
The functions add_two, add_three and add_whatever above are all 
closures. That's a term that comes from computer science, and its exact 
definition isn't important, but the main thing is that in practice a 
closure is a function which gains at least one variable from the 
enclosing outer function that creates it. In the case of the various 
add_* functions, the inner function plus() requires a value for variable 
n, which it gets from enclosing adder_factory() scope.

Closures are advanced but *incredibly* useful in Python. In languages 
like Java where functions are not first-class values, you have to do 
something like this instead:

class Adder:
    def __init__(self, n):
        self.n = n
    def __call__(self, arg):
        return arg + self.n

add_two = Adder(2)
add_three = Adder(3)


[more to follow]


-- 
Steven