[Persistence-sig] A simple Observation API

[Phillip J. Eby]

I've taken the time this evening to draft a simple Observation API, and an 
implementation of it.  It's not well-documented, but the API should be 
fairly clear from the example below.  Comments and questions encouraged.

Note that this draft doesn't deal with any threading issues whatsoever.  It 
also doesn't try to address the possibility that an observer might throw an 
exception when it's given a notification during a 'finally' clause that 
closes a beginWrite/endWrite pair.  If anybody has suggestions for how to 
handle these situations, please let me know.

By the way, my informal tests show that subclassing Observable makes an 
object's attribute read access approximately 11 times slower than normal, 
even if no actual observation is taking place (i.e., an _o_readHook is not 
set).  I have not yet done a timing comparison for write operations and 
method calls, but I expect the slowdown to be as bad, or worse.  Rewriting 
Observation.py in C, using structure slots for many of the attributes, 
would probably eliminate most of these slowdowns, at least for unobserved 
instances.  Of course, any operations actually performed by a change 
observer or read hook, would add their own overhead, in addition to the raw 
observation overhead.

This is a fairly "transparent" API, although it still requires the user to 
subclass a specific base, and declare which mutable attributes are touched 
by what methods.  But it is less invasive, in that observation-specific 
code does not need to be incorporated into the methods themselves.

One possible enhancement to this framework: use separate observer lists for 
the beforeChange() and afterChange() events, and make them simple callables 
instead of objects with obvservation methods.  While this would require an 
additional attribute, it would simplify the process of creating dynamic 
activation methods, and reduce calls in situations where only one event 
needed to be captured.  This could be useful for setting up observation on 
a mutable attribute so as to "wire" it to trigger change events on the 
object(s) that contained it.

Anyway, here's the demo, followed by the module itself.


#### Demo of observation API ####

from Observation import Observable, WritingMethod

class aSubject(Observable):

     def __init__(self):
         self.spam = []

     # __init__ touches spam, but shouldn't notify anyone about it
     __init__ = WritingMethod(__init__, ignore=['spam'])


     def addSpam(self,spam):
         self.spam.append(spam)

     # addSpam touches spam, even though it doesn't set the attribute
     addSpam = WritingMethod(addSpam, attrs=['spam'])


     def setFoo(self, foo):
         self.foo = foo
         self.bar = 3*foo

     # setFoo modifies multiple attributes, and should send at most
     # one notice of modification, upon exiting.
     setFoo = WritingMethod(setFoo)


class anObserver(object):

     def beforeChange(self, ob):
         print ob,"is about to change"

     def afterChange(self, ob, attrs):
         print ob,"changed",attrs

     def getAttr(self, ob, attr):
         print "reading",attr,"of",ob
         return object.__getattribute__(ob,attr)


subj = aSubject()
obs  = anObserver()

subj._o_changeObservers = (obs,)
subj._o_readHook = obs.getAttr

subj.setFoo(9)
print subj.bar

subj.addSpam('1 can')

#####  End sample code #####



#### Observation.py ####

__all__ = ['Observable', 'WritingMethod', 'getAttr', 'setAttr', 'delAttr']

getAttr = object.__getattribute__
setAttr = object.__setattr__
delAttr = object.__delattr__

class Observable(object):

     """Object that can send read/write notifications"""

     _o_readHook = staticmethod(getAttr)
     _o_nestCount = 0
     _o_changedAttrs = ()
     _o_observers = ()

     def _o_beginWrite(self):

         """Start a (possibly nested) write operation"""

         ct = self._o_nestCount
         self._o_nestCount = ct + 1

         if ct:
             return

         for ob in self._o_changeObservers:
             ob.beforeChange(self)


     def _o_endWrite(self):

         """Finish a (possibly nested) write operation"""

         ct = self._o_nestCount = self._o_nestCount - 1

         if ct:
             return

         ca = self._o_changedAttrs

         if ca:
             del self._o_changedAttrs
             for ob in self._o_changeObservers:
                 ob.afterChange(self,ca)


     def __getattribute__(self,attr):

         """Return an attribute of the object, using a read hook if 
available"""

         if attr.startswith('_o_') or attr=='__dict__':
             return getAttr(self,attr)

         return getAttr(self,'_o_readHook')(self, attr)


     def __setattr__(self,attr,val):

         if attr.startswith('_o_') or attr=='__dict__':
             setAttr(self,attr,val)

         else:
             self._o_beginWrite()

             try:
                 ca = self._o_changedAttrs

                 if attr not in ca:
                     self._o_changedAttrs = ca + (attr,)

                 setAttr(self,attr,val)

             finally:
                 self._o_endWrite()


     def __delattr__(self,attr):

         if attr.startswith('_o_') or attr=='__dict__':
             delAttr(self,attr)

         else:
             self._o_beginWrite()

             try:
                 ca = self._o_changedAttrs
                 if attr not in ca:
                     self._o_changedAttrs = ca + (attr,)

                 delAttr(self,attr)
             finally:
                 self._o_endWrite()


from new import instancemethod

class WritingMethod(object):

     """Wrap this around a function to handle write observation 
automagically"""

     def __init__(self, func, attrs=(), ignore=()):
         self.func = func
         self.attrs = tuple(attrs)
         self.ignore = tuple(ignore)

     def __get__(self, ob, typ=None):

         if typ is None:
            typ = type(ob)

         return instancemethod(self, ob, typ)

     def __call__(self, inst, *args, **kwargs):

         attrs, remove = self.attrs, self.ignore

         inst._o_beginWrite()

         try:
             if attrs or remove:
                 ca = inst._o_changedAttrs
                 remove = [(r,1) for r in remove if r not in ca]
                 inst._o_changedAttrs = ca + attrs

             return self.func(inst, *args, **kwargs)

         finally:
             if remove:
                 inst._o_changedAttrs = tuple(
                     [a for a in inst._o_changedAttrs if a not in remove]
                 )
             inst._o_endWrite()