Function decorator that caches function results

[Paul Rubin]

Steven D'Aprano <steve at REMOVETHIScyber.com.au> writes:
> Is it correct to say that Python *always* creates a closure whenever a def
> or lambda is executed? Or is it only for *certain* defs/lambdas?

The word closure is being used two different ways.  First of all the
computer-science term "closure" which means a combination of a
function and its lexical environment.  Second of all the data field
named "closure" which lives inside certain function objects in Python.
The second one is created only in certain Python functions but that's
an implementation detail.  It was added because in the old days,
Python didn't support closures at all, so implementing them was done
straightforwardly by adding a field that contains the closure
environment, for Python functions that needed it.

So the answer to your first question above is "always" in the CS sense
and "only for certain defs/lambdas" in the implementation sense.

> Is it correct to talk about functions *having* closures, or that they
> *are* closures, or that the function is *part of* a closure?

The function is part of the closure.  I think you're a little confused
by implementation issues.  In Python, functions are first-class
objects, like class instances or strings.  If you say 
  x = 'foo' + 'bar'
a new string is created out of nowhere.  If you say
  def f(x): return x+3
a new function is created out of nowhere.  Lambda notation maybe
shows more clearly that functions are values:
  f = (lambda x: x+3)
is the same as the "def f" above.

Let's go back a step.  Do you understand what a free variable is?
Let's say:
  def f():
     print x
f refers to x but x is not defined in f.  We say x is "free" in f.

In the old days, this could only mean one thing: x was a global
variable.  Python didn't have nested scopes.  If a variable wasn't
found in a function's locals, it could only come from the globals
that are shared across all functions.

Python now has nested scopes.  That means x doesn't have to come
from the globals, it can be a local from some surrounding function:
  def g(n):
     x = n   # x is local to g
     def f():
        print x
     f()
When you run g(3), Python prints 3.  That's not too confusing, right?

Suppose instead of calling f, you return it.  You can do that in Python:
  def g(n):
     x = n   # x is local to g
     def f():
        print x
     return f

  h3 = g(3)   # h3 is the function g creates when you run it

What will happen when you call h3()?  It prints 3.  How did it know to
do that?  Because h3 doesn't just contain executable bytecode, it also
contains f's lexical environment from when g ran.  (f's lexical
environment means the values of f's free variables from outer scopes
at the time f was created).  Now suppose you say:

  h5 = g(5)   # make a new function

See, h3 and h5 are two different functions:

  h3()  # still prints 3
  h5()  # prints 5

They contain the same executable code, but different lexical environments.

The combination of executable code plus lexical environment is called a
closure (CS term).  In Python, every function is a (CS) closure.  The
environment lives in a field called func_closure or something like that.

However, when there are no free variables, Python doesn't bother
creating that field.  That was ALWAYS the case in old versions of
Python.  Python didn't recognize nested scopes; there were only locals
and globals.  

> If all functions are closures, what does it mean to say that some
> languages (e.g. Java, C++, C#) don't have closures but merely
> simulate them? Are there any languages that don't have closures?

Those languages listed there don't have closures.  They don't even
have first-class functions.  Imagine what the h3 and h5 examples from
above would do in those languages.

To be fair, there's an argument that closures are un-Pythonic: you can
use class instances instead.  You could say:

   class G:
      def __init__(self, n): self.x = n
      def __call__(self): return self.x
   h3 = G(3)
   h5 = G(5)

and h3() prints 3 and h5() prints 5, just like the corresponding
closures.  But now you've polluted the name space with this extra
class G, and (depending on the situation) maybe you've written more
cumbersome code.  It's partly a matter of taste and preferences.  To
Lisp and Scheme users, closures are very natural.  To OOP programmers,
maybe they're confusing and unnecessary.