Feature request: subclassing FunctionType [Was: Some language proposals]

[Jacek Generowicz]

michele.simionato at poste.it (Michele Simionato) writes:

> I don't know Common Lisp, but Scheme scope rules do what I expect:
> 
> (define (make-adders n)
>   (unfold (cut = <> n) (lambda (i) (lambda (x) (+ x i))) add1 0))
> 
> (define-values (add0 add1) (apply values (make-adders 2)))
> 
> (print (add0 0)) ;=> 0
> (print (add1 0)) ;=> 1

I think that you are misleading yourself. You are comparing apples to
quinces.

Let's try to reproduce your Scheme code in Python. (I'm no Schemer,
but here goes ...).  Googling for "scheme unfold" comes up with
something which I can fake in Python more or less like this:

def unfold(p, f, g, seed):
    if p(seed):
        return []
    return [f(seed)] + unfold(p, f, g, g(seed))

I'm guessing that add1 does this:

def scheme_add1(n): # don't name-clash with the add1 we're about to
make
    return n+1

Allow me to simplify cut to the following, which I hope you will agree
does what you want in this particular case:

def cut(limit): # [*]
    def cut(x):
        return not x<limit
    return cut

Now our user code becomes

def make_adders(n):
    return unfold(cut(n), lambda i: lambda x:x+i, scheme_add1, 0)

add0, add1 = make_adders(2)

print add0(0) # => 0
print add1(0) # => 1

Note: Python gives the same results as Scheme.

So, what's going on? The original problem was that all the closures
that you generated in your Python code shared a single binding for
their free variable. The solution is to create an inner binding unique
to each closure, which shadows the original shared binding. In your
Python solution you achieve this by creating a local variable i, which
shadows the outer i (so the i in the lambda is no longer free, and the
lambda is no longer a closure); in my CL solution I achieve this by
creating a new binding of i using let (which is still outside of the
scope of the lambda, so lambda's i remains free, and therefore it
remains a closure). In your Scheme solution, and my Python translation
of it, you'll see that we have:

  lambda i: lambda x:x+i

which gets called by unfold to create the closure you want. Note that
you are rebinding i in the outer lambda, each time unfold calls it to
create a closure over the next number in the sequence, and that
binding is visible only to the single closure being returned from that
invocation of the outer lambda. So, to compare like to like, you
should do the same in your original Python attempt.

def make_adders(n):
    return [ (lambda i: lambda x:x+i)(i) for i in range(n) ]

add0, add1 = make_adders(2)
print add0(0) # => 0
print add1(0) # => 1

So, you see, Python's scoping rules are just like Scheme's, in this
sense. Unfortunately, I don't think that it's so easy (or even
possible?) to demonstrate it the other way around (ie show a Scheme
snippet in which add0(0) == add1(0) == 1), because of Scheme's
insistence on using recursion to implement iteration. You see, in both
the Python and the Common Lisp versions of the original, there is a
single biding of i in the loop, which gets modified as the loop goes
about its business. In Scheme, the "loop variable" gets rebound each
time the looping function (eg unfold) calls itself, so for each and
every number in the sequence there is a seperate binding of seed, as
well as a separate binding of i.

Looking at it pictorially (variables with a * next to them are free,
those with a + are bound; where the value of the binding does not
change, the value is shown in parentheses; arrow up means binding
found by free variable, arrow down means binding by argument passing):

The Python case

    list comprehension: i+
                        ^
                        |
              ________ / \___
            /                \
     add0:  i*         add1:  i*


Scheme style case

  unfold: seed+ (0)
             |
             |
            / \
           |   ---(via scheme_add1)--
           |                          \
           V                           V
   lambda: i+ (0)           unfold: seed+ (1)
           ^                           |
           |                           V
     add0: i*                  lambda: i+ (1)
                                       ^
                                       |
                                 add1: i*

The scoping rules in the two languages are pretty much the same, but
the scheme code throws in lots of extra scopes which are absent from
the (original) Python code.

If you can find a way of making a genuine loop in Scheme, you should
observe similar behaviour to that of the original Python example.

> However, now that I see that there are languages that implement the
> scope rule I would expect

I hope that you agree, by now, that Scheme is _not_ an example of such
a language.

If you need more convincing, how about the following?

guile> (define (make-adder i)
  (list (lambda (x) (+ x i))
        (lambda (n) (set! i (+ i n)))))
guile> (define tmp (make-adder 5))
guile> (define add5 (car tmp))
guile> (define hehe (cadr tmp))
guile> (add5 3)
8
guile> (hehe 1000)
guile> (add5 3)
1008

Conclusion: the value of i that add5 sees, is definitely _not_ fixed
at the time of creation of add5.

> and since I don't see a real life case where Python default of using
> the value of "i" as the usage time is useful,

You are effectively saying that you don't see the point of using the
run-time value of a variable, and that you'd rather be stuck with the
first value that was ever assigned to the variable. You'd never claim
this was the right thing for bound variables (including globals
referenced in a local scope), would you? And I don't believe that you
_really_ think it is appropriate for free variables either.

Try the following:

x = 3
def show(): global x; print x
show() # => 3
x += 1
show() # => 4

def outer():
    x = 3
    def show(): print x
    show() # => 3
    x += 1
    show() # => 4
outer()

... which prints out the numbers 3,4,3 and 4.

You don't really believe that the last number should be 3, do you ?

[In case anyone is in any doubt, the following is essentially
identical

def outer(x):
    def show(): print x
    show() # => 3
    x += 1
    show() # => 4
outer(3)
]

[*] def cut(limit):
        def cut(x):
            return not x<limit
        return cut

Sorry, I should have written cut as follows, which is much clearer and
more Pythonic, of course

    class cut:
    
        def __init__(self,limit):
            self.limit = limit
    
        def __call__(self,x):
            return not x<limit

:-)