aspect-oriented demo using metaclasses

[Mark McEahern]

Hi, I have tried and failed several times to understand metaclasses.  This
morning, I trudged up the learning curve a little further than I have before
and I wanted to share what I discovered.  I've been intrigued by the idea of
using metaclasses as a way to implement aspect-oriented programming (AOP) in
Python.  To me, the promise of AOP is you should ideally write your classes
in complete ignorance of the aspects that you want to plug in later.
Tracing is probably the simplest example of an aspect I can think of.
Persistence is a more complicated aspect (to me, anyway).  So I figured I'd
start with tracing.

I owe much to previous threads on AOP, in particular Pedro Rodriguez's
non-metaclass implementation:


http://groups.google.com/groups?selm=pan.2002.01.13.11.22.54.248401.9857%40c
lub-internet.fr

This is written in the style of a tutorial, but admittedly, it's not very
polished.  I would appreciate any and all comments, criticism, feedback,
suggestions, amplifications, discussion, etc.

**

A simple AOP framework in Python using metaclasses

Let's start with a simple class:

<code>
class foo:

    def bar(self, *args, **kwargs):
        pass

f = foo()
f.bar()
</code>

As expected, this doesn't do anything.  Suppose I want to be notified
before and after the bar method is called.  (The wording of that is
purposefully vague.)  And I don't want to have to modify the foo class
at all in order to do this.  I want to intervene in the creation of
this class (not just instances of the class) and insert pre and post
methods for each function.

Let's start with a simple metaclass definition that doesn't do
anything:

<code>
class aspect(type):

    def __init__(cls, name, bases, dict):
        super(aspect, cls).__init__(name, bases, dict)

__metaclass__ = aspect

class foo:

    def bar(self, *args, **kwargs):
        pass

f = foo()
f.bar()
</code>

Again, this doesn't have any noticeable effect.  Let's insert some
print statements to see what's happening though:

<code>
class aspect(type):

    def __init__(cls, name, bases, dict):
        super(aspect, cls).__init__(name, bases, dict)
        for k,v in dict.items():
            print "type(%s) --> %s" % (k, type(v))

__metaclass__ = aspect

print "before class foo:"

class foo:

    def bar(self, *args, **kwargs):
        pass

print "before f = foo()"

f = foo()
f.bar()
</code>

<output>
before class foo:
type(__module__) --> <type 'str'>
type(bar) --> <type 'function'>
before f = foo()
</output>

This shows us that the aspect metaclass' __init__ gets called as a
result of declaring our foo class, before any instance's of foo are
created.

We want to iterate through the class's dict and do something for each
method (I'm not worried about distinguishing staticmethod and
classmethod type methods for now):

<code>
class Aspect(type):

    def __init__(cls, name, bases, dict):
        super(Aspect, cls).__init__(name, bases, dict)
        for k,v in dict.items():
            if type(v) == type(lambda x:x):
                print "%s is a function." % k

__metaclass__ = Aspect

class foo:

    def bar(self, *args, **kwargs):
        print "real bar(%s, %s)" % (args, kwargs)

f = foo()
f.bar("a", "b", foo="c")
</code>

<output>
bar is a function.
real bar(('a', 'b'), {'foo': 'c'})
</output>

Rather than importing types and using types.FunctionType, I just
compare the type of the item to the type of an anonymous function (via
lambda).  Is that cool or what?  ;-)

Also, notice that I replaced pass in the body of bar() with a simple
print statement that tells me how the real bar() is being called.
That will be useful later.

For each method (or function), I want to wrap that method so that I
get notified when it's called.  So I created a wrapped_method class
that is somewhat (exactly?) like a field descriptor (e.g., property):

<code>
class wrapped_method(object):

    def __init__(self, cls, method):
        self.class_name = cls.__name__
        self.method = method
        self.method_name = method.func_name
        self.observers = []

    def __call__(self, *args, **kwargs):
        self.before(*args, **kwargs)
        self.method(self, *args, **kwargs)
        self.after(*args, **kwargs)

    def before(self, *args, **kwargs):
        print "before %s.%s(%s, %s)" % (self.class_name, self.method_name,
                                        args, kwargs)

    def after(self, *args, **kwargs):
        print "after %s.%s(%s, %s)" % (self.class_name, self.method_name,
                                       args, kwargs)

class Aspect(type):

    def __init__(cls, name, bases, dict):
        super(Aspect, cls).__init__(name, bases, dict)
        for k,v in dict.items():
            if type(v) == type(lambda x:x):
                setattr(cls, k, wrapped_method(cls, v))

__metaclass__ = Aspect

class foo:

    def bar(self, *args, **kwargs):
        print "real bar(%s, %s)" % (args, kwargs)

f = foo()
f.bar("a", "b", foo="c")
</code>

<output>
before foo.bar(('a', 'b'), {'foo': 'c'})
real bar(('a', 'b'), {'foo': 'c'})
after foo.bar(('a', 'b'), {'foo': 'c'})
</output>

Now we're starting to get somewhere!  The next step is to create a
framework where multiple observers can plug into the before and after
events for any given method call.  Rather than taking more baby steps
to get there, this is the complete implementation I have so far:

<code>
import sys

class wrapped_method(object):

    def __init__(self, cls, method):
        self.class_name = cls.__name__
        self.method = method
        self.method_name = method.func_name
        self.observers = []

    def __call__(self, *args, **kwargs):
        self.before(*args, **kwargs)
        self.method(self, *args, **kwargs)
        self.after(*args, **kwargs)

    def before(self, *args, **kwargs):
        for o in self.observers:
            o.before(self.class_name, self.method_name, *args, **kwargs)

    def after(self, *args, **kwargs):
        for o in self.observers:
            o.after(self.class_name, self.method_name, *args, **kwargs)

    def notify_me(self, observer):
        self.observers.append(observer)

class VirtualClassError(Exception):pass

class Observer:

    def __init__(self, *args, **kwargs):
        if self.__class__ is Observer:
            raise VirtualClassError(self.__class__.__name__)

    def before(self, class_name, method_name, *args, **kwargs):
        pass

    def after(self, class_name, method_name, *args, **kwargs):
        pass

class Trace(Observer):

    def __init__(self, filename=None):
        self.filename = filename

    def write(self, prefix, class_name, method_name, *args, **kwargs):
        cls = class_name
        s = "%s: %s.%s(%s, %s)" % (prefix, cls, method_name, args, kwargs)
        if not self.filename:
            f = sys.stdout
        else:
            f = file(self.filename)
        f.write(s)
        f.write("\n")
        if self.filename:
            f.close()

    def before(self, class_name, method_name, *args, **kwargs):
        self.write("before", class_name, method_name, *args, **kwargs)

    def after(self, class_name, method_name, *args, **kwargs):
        self.write("after", class_name, method_name, *args, **kwargs)

class Aspect(type):

    def __init__(cls, name, bases, dict):
        super(Aspect, cls).__init__(name, bases, dict)
        for k,v in dict.items():
            if type(v) == type(lambda x:x):
                setattr(cls, k, wrapped_method(cls, v))

__metaclass__ = Aspect

class foo:

    def bar(self, *args, **kwargs):
        print "real bar(%s, %s)" % (args, kwargs)

def main():
    foo.bar.notify_me(Trace())
    f = foo()
    f.bar("a", "b", foo="c")

if __name__ == "__main__":
    main()
</code>

<output>
before: foo.bar(('a', 'b'), {'foo': 'c'})
real bar(('a', 'b'), {'foo': 'c'})
after: foo.bar(('a', 'b'), {'foo': 'c'})
</output>

The output is the same, but it's actually being generated by an
instance of the Trace class.

************
Observations
************

Obviously, Trace is designed so that I can specify a filename when
creating an instance of it and that would use that file rather than
sys.stdout.

Observers should probably be able to register for particular events
(i.e., just before).  They should be able to unregister.  It should
also be possible to register for some classes, some methods, and not
others.  What is a good way to write the rules for enrolling
in notifications?  (And what's the word for this that AspectJ uses?)

I explicitly pass self from the wrapped_method instance to the actual
wrapped method.  However, this is not really the instance of the
class, it's the instance of the wrapped_method object.  How do I pass
the actual instance to the wrapped method--whether explicitly or
implicitly?

I should probably explicitly set the metaclass for wrapped_method,
Trace, Observer, and even VirtualClassError to "type" (the default
metaclass) to avoid weird infinite loops where the members of the
aspect framework are themselves being aspected.

I need to think more about how this would be used.  My vague
impression of AspectJ is that you use it from an IDE, specifying
different conditions for filtering classes and methods (what's the
word for that again?).  When you compile with aspect support turned
on, the generated code has the aspects weaved in.  What are the
equivalent developer use cases for weaving aspects into Python code?
The whole point it seems to me is that when I author the classes to be
aspected, I shouldn't have to do a thing--no subclassing, no method
definition.  I just write my classes, omitting entirely any
consideration of the future aspects to be weaved in.  I mean, that's
the whole point, right?  To simplify the code.

In the example, foo doesn't know diddly about the fact that it's being
traced.  And we could turn that tracing on or off simply by changing
the module-level __metaclass__ variable.

I should probably and try:...except:pass to the framework so that it
doesn't generate errors.  During development of the framework itself,
of course, I want to know about errors.

Observer is an interface-type base class.  It doesn't provide
implementation.  It could easily be omitted, but it servers to
document what Trace and other potential observers need to look like.

It would be easy to add aspects for error handling.  This is a very
quick and dirty example for error notification:

In wrapped_method, I would change __call__ like this:

    def __call__(self, *args, **kwargs):
        self.before(*args, **kwargs)
        try:
            self.method(self, *args, **kwargs)
        except Exception, e:
            self.on_error(e, *args, **kwargs)
            raise
        self.after(*args, **kwargs)  Rather than using sys.stdout, I could
have specified a file.



and add on on_error() method to wrapped_method:

    def on_error(self, error, *args, **kwargs):
        for o in self.observers:
            o.on_error(error, self.class_name, self.method_name,
                       *args, **kwargs)

Observer:

    def on_error(self, error, class_name, method_name, *args, **kwargs):
        pass

and Trace:

    def on_error(self, error, class_name, method_name, *args, **kwargs):
        self.write("error %s" % error, class_name, method_name, *args,
                   **kwargs)

-