Confused with Functions and decorators

[Steven D'Aprano]

On Sat, 19 Jul 2014 03:52:18 -0700, Jerry lu wrote:

> Ok so i am trying to learn this and i do not understand some of it. I
> also tried to searched the web but i couldnt find any answers.
> 
> 1. I dont understand when i will need to use a function that returns
> another function. eg
>                    def outer():
>                          def inner():
>                                x = 5
>                           return inner
>     
>      why not just use inner and forget about the whole outer function?

For the example given, there's no point in using an inner function. 
(Especially since the inner function does nothing useful.) But there are 
three common reasons for writing inner functions:

#1 Encapsulation and information hiding.

If you've ever programmed in Pascal, this may be familiar to you. You use 
inner functions to hide them away from code outside of the function. Here 
is a toy example that shows the basic idea:

def function(x):
    def increment(x):
        return x + 1
    y = increment(x)
    print x, y


In Python, I would rarely bother to write code like that. I'd just make 
the increment function a top-level function, marked as "private" with a 
leading underscore: _increment.

One place where the Pascal-style nested function may be useful is with 
recursive functions:

def factorial(n):
    if not isinstance(n, int):
        raise TypeError("n must be an integer")
    if not n >= 0:
        raise ValueError("n must be zero or positive")
    def inner_fact(n):
        if n <= 1:
            return 1
        return n*inner_fact(n-1)
    return inner_fact(n)


This has three advantages:

* The caller cannot (easily) bypass the outer function and call the inner 
function directly.

* The outer function performs all the argument checking and any pre-
processing once only, the inner function can safely assume the argument 
is valid and eschew pointless error checking.

* The outer function can be renamed without breaking the recursion.



#2 Don't Repeat Yourself (DRY)

Sometimes you may have a big function that performs the same chunk of 
code in two places. For example, you might write a function which 
processes a file, and you want to accept either an open file object or a 
file name:

def process(the_file):
    def do_stuff(fp):
        for line in fp:
            print line
    if isinstance(the_file, str):
        with open(the_file, 'r') as f:
            do_stuff(f)
    else:
        do_stuff(the_file)


Again, it is common to put do_stuff as a private top-level function, but 
if you want to hide it away as an internal function, you can.


#3 Closures and factory functions

But now we come to the most important reason to use internal functions. 
All of the inner functions I've shown so far have been ordinary functions 
that merely happened to be nested inside another function. But when we 
come to closures, that is not the case.

A closure is an inner function which "remembers" the state of its 
environment when called. Here's a basic example:

def make_incrementer(inc):
    """Make an new function which increments its argument by inc.

    Examples of use:

    >>> add_one = make_incrementer(1)
    >>> add_two = make_incrementer(2)
    >>> add_one(7)
    8
    >>> add_two(7)
    9

    """
    def incrementer(x):
        return x + inc
    return incrementer


This takes more effort to explain than to write!

The "make_incrementer" function is a *factory function*. It builds a new 
function each time it is called, then returns it. What does this new, 
inner, function do? It takes a single argument, x, and returns x + inc.

Where does the inner function get the value of inc from? This is what 
makes it a closure -- it gets the value of inc from the outer function, 
the factory. How it works is something like like this:

make_incrementer(3)

=> build a function "incrementer" which takes a single argument x
=> take a snap-shot of the state of make_incrementer function
   which includes inc=3
=> stuff that snap-shot into the incrementer function
=> return the incrementer function


Now, whenever you call that newly returned function, it will always see 
its own private snap-shot that includes inc=3.

The use of closures and factory functions is the most common, and 
powerful, use for inner functions. 9 times out of 10, if you see a 
decoratored function:

@decorator
def func(arg):
    pass


the decorator is a factory function which takes an function as argument, 
and returns a new function which which includes the old function inside 
the closure. But that's now getting into more advanced territory.



> 2. This is more yes or no question when you pass in a func into another
> func as a parameter do you need to specify the parameter that the func
> is being passed into as func?

No.

You can pass functions around from place to place until the cows come 
how, stuff them into lists and dicts, extract them again, and not care 
one iota about its arguments until you actually call it.


-- 
Steven