Feature request: subclassing FunctionType [Was: Some language proposals]

[Michele Simionato]

I have read with interest the recent thread about closures. The funny
thing is that the authors are arguing one against the other but I
actually agree with all of them and I have a proposal that may
be of some interest. To refresh your memory, I report here some 
quotation from that thread.

Jacek Generowicz:
 
> I sumbit to you that read-only closures are rare in Python because
> they are a recent addition to the language.
> 
> I submit to you that read-write closures are so much rarer still,
> because they require an ugly hack to work.

Paul Prescod:

> I disagree. Closures are rare in Python because Python is primarily an 
> OOP language.

Micheal Hudson:

> Using closures to fake objects sucks.

Jacek Generowicz:

> When I need "objects" I use objects, and when I need closures I want
> to be able to use closures.
> [...] I have some stateful methods of classes (which I create
> dynamically and stick onto the class as the information about the
> existence of the method becomes available). By implementing them as
> closures I can just stick them on to the class. If I were to implement
> them as instances then I'd have to reimplement all the descriptors
> that take care of turning functions into bound or unbound methods.
> 
> [At this point I'd love someone to step up and show me how to re-use
> the existing FunctionType descriptors in my own classes.]

> Another example. I make quite heavy use of a trivial memoizer. The
> closure verison is much shorter, clearer and faster. 

Paul Prescod:

> Before Python had nested scopes at all, it was VERY common to fake them 
> using a linguistic trick: default arguments. This idiom was very common 
> in Python's source base. Python didn't have a proper way to do the 
> nesting but people figured out how to do it anyway and did it a LOT. 
> (similarly people often emulate OO in C)

> The "wrap your mutables" feature is by comparison _much less intrusive_ 
> but I don't ever see it in real Python code. This suggests to me that 
> people don't need the feature that badly.

There is some merit on all those points of view. Speaking for myself,
I would certainly use more closures if they were better supported by 
the language. The fact that closures are read-only don't bothers me too 
much, since I also think that when you want write access to a closure 
you are really trying to fake an object, and you are better off using
a real object. On the other hand, I am really bothered by the scoping
rules. My first post on this mailing list some time ago was caused by
problems I had with the surprising scope rules. Take for instance
this example:

def make_adders(n):
    return [(lambda x: x+i) for i in range(n)]

add0,add1=make_adders(2)

print add0(0) #=> 1
print add1(0) #=> 1

This does not work as expected, and the solution is an ugly hack
with default arguments:

def make_adders(n):
    return [(lambda x,i=i: x+i) for i in range(n)]

add0,add1=make_adders(2)

print add0(0) #=> 0
print add1(0) #=> 1

I don't like that, but I don't think that the current scope rules
will change any soon. Alternatively, I can use a class:

class adder(object):
    def __init__(self,i):
        self.i=i
    def __call__(self,x):
        return x+self.i

def make_adders(n):
    return [adder(i) for i in range(n)]

add0,add1=make_adders(2)

print add0(0) #=> 0
print add1(0) #=> 1

However, this is not really a solution, since as Jacek pointed
out, to really fake a function I need to reimplement the
descriptor protocol and I cannot get it from FunctionType. 
I have experience with the examples he cites, memoizing 
functions and wrapping methods, and they are exactly the 
examples where I have used closures instead of callable objects.
OTOH, Paul Prescod is right and Python is primarily an 
OOP language, so I would prefer to use real objects over
closures.

What I think would help is the ability to subclass FunctionType.
I remember a thread of some time ago where Tim Peters said
that it is quite doable, simply nobody bothered to implement 
it yet.

At the moment, I can fake the built-in FunctionType with something like
that:

class function(object):
    def __init__(self,func):
        self.__func__=func
    def __call__(self,*args,**kw):
        return self.__func__(*args,**kw)
    def __get__(self,obj,cls=None):
        return self.__func__.__get__(obj,cls)

Then I can subclass "function" to implement any kind of wrapped functions.
For instance, to trace functions:

class traced_function(function):
    def __init__(self,func):
        def traced_func(*args,**kw): # using an handy closure ...
            print "Calling %s with arguments %s %s ..." % (func.__name__,args,kw)
            result=func(*args,**kw)
            print "Called %s with result %s" % (func.__name__,result)
            return result
        self.__func__=traced_func

Since "function" implements the descriptor protocol, I can wrap methods
for free:

class C(object):
    add1=traced_function(lambda self,x: x+1)

C().add1(1)

My previous example with the adder would read

class adder(function):
    def __init__(self,i):
        self.__func__=lambda x: x+i

I think this reads quite well. At the present, however, I cannot subclass
FunctionType and I am a bit disturbed by that, also because I can subclass 
the other built-ins types and there no reason why functions must be so
special: at they end they are just callable objects with a descriptor
protocol. So, I submit it as a feature request for Python 2.4.
Maybe, if enough people wants the feature, we may get it! 

        Michele Simionato