super(..) calling __getattr__ (was: RE: Question about accessing class-attributes.)

[Bjorn Pettersen]

> From: Alex Martelli [mailto:aleax at aleax.it] 
> From: Alex Martelli [mailto:aleax at aleax.it] 
> 
> On Wednesday 30 April 2003 02:15 am, Bjorn Pettersen wrote:
>    ...
> > > is probably some sound implementation reason for the current
> > > behavior, but if so it should be better documented, I think.
> >
> > SF 729913. I have not been able to find an exact description of the
> > lookup rules for new-style classes anywhere.
> 
> Good point.  I guess the language reference DOES need to provide
> "language lawyers" with those!

I guess I deserved that <grin>. Seriously though, since I'm mandated to
learn C# right now, I figured I might as well use it to create a
translator from a very small subset of Python to C# (going to IL seems
like overkill :-)  To do that however, I'd need to know how to implement
a trivial thing like attribute lookup :-)  [Yes, I know, I should just
drag the pretty pictures and double click on them <wink>].

I'm following your suggestion and breaking this into two issues.
__getattr__ is a special case. (Just to get us all down to the nice
low-level details :-) Normally x.foo() will try:

  fn = type(x).foo.__get__(x, type(x))
  fn()

if the first line fails with an AttributeError, the following scheme is
tried:

  v = type(x).__getattr__(x, 'foo')
  v()

> > > > Perhaps a more realistic exmaple?:
> > > >
> > > >   class Secure(object):
> > > >     def __init__(self, name):
> > > >       self.name = name
> > > >
> > > >     def __getattr__(self, attr):
> > > >       return getSecureMethod(
> > > >                  self.name, 
> > > >                  attr, 
> > > >                  getUserPrivileges())
> > > >
> > > >   class LogFoo(Secure):
> > > >     def __init__(self):
> > > >       Secure.__init__(self, 'SalaryDB')
> > > >
> > > >     def foo(self):
> > > >       print 'calling foo'
> > > >       return super(LogFoo, self).foo()
> > > >
> > > >   lf = LogFoo()
> > > >   print lf.foo()
> > >
> > > Sorry, I don't get it.  How is this related to metaclasses...?
> >
> > Not terribly obvious was it... :-)  And it's a bad example too. 
> > The thinking was that if Super's metaclass could intercept 
> > special methods through it's __getattr__, the object returned 
> > from super() could look more like a regular object.  That 
> > Super.__getattr__ doesn't check for a __getattr__ in the 
> > __mro__ chain is a problem, but a  separate one (sorry
> > for the confustion).
> 
> "the object returned from super" DOES look a lot "like a regular
> object", except it has a custom metaclass that ensures its fundamental
> behavior -- attribute lookups starts AFTER the given class,

Actually, it looks like just a regular class __getattr__ method, this
one just mucking around with the __mro__ of two objects passed to
__init__ to figure out what to return. (There can be a __getattr__ in a
metaclass too, but that's unrelated to this example).

[..]
> if and when any bound-methods are built, the instance they're 
> bound to is the given one.  We can't break THAT, I think.

Of course. (I must be missing your point?)

> I think that, currently, if you want the functionality you seem to be
> after in this example, you must ensure that superclass Secure has,
> in its class dict at the time type.new is called to build it, explicit
> entries for all specialmethods of your interest -- that way, the slots
> will be filled in (perhaps with special-purpose descriptors that do
> the just-in-time lookup you may want).  But, for methods that you
> look up explicitly, a metaclass's __getattr__ (NOT of course the
> __getattr__ of the CLASS on which you're looking up -- that only
> applies to lookups on INSTANCES of that class) should work fine.

Yes, that's why this was a bad example. __getattr__ isn't a "special
method" (like len, str, abs, etc.), however it is special in that it's
special cased in the attribute lookup algorithm when the regular path
turns up empty.

So really, we don't need to think about the class of super(..) in this
instance at all, we just need to ask the individual classes on the mro
if they can handle attribute 'foo' through their __getattr__ if they
can't from their regular lookup. Modified Super (index can throw, but I
think it illustrates what's going on better :-)

  class Super(object):
     def __init__(self, type, obj=None):
        self.__type__ = type
        self.__obj__ = obj
  
     def __get__(self, obj, type=None):
        if self.__obj__ is None and obj is not None:
           return Super(self.__type__, obj)
        else:
           return self
           
     def __getattr__(self, attr):
        if isinstance(self.__obj__, self.__type__):
           mro = list(self.__obj__.__class__.__mro__)
        else:
           mro = list(self.__obj__.__mro__)
  
        mro = mro[mro.index(self.__type__) + 1:]
  
        for cls in mro:
           if attr in cls.__dict__:
              x = cls.__dict__[attr]
              if hasattr(x, '__get__'):
                 return x.__get__(self.__obj__)
              else:
                 return x
              
           if hasattr(cls, '__getattr__'):
              try:
                 return cls.__getattr__(self.__obj__, attr)
              except AttributeError:
                 pass
              
        raise AttributeError, attr

I've tested it both on the classic diamond, and on:

   class BB(object):
      def __getattr__(self, attr):
         return lambda:attr
   
   class B(BB):
      def foo(self): print 'B.foo'
      
      def __getattr__(self, attr):
         if attr == 'bar':
            print 'B.__getattr__("bar")'
            return lambda:42
         else:
            raise AttributeError(attr)
      
   class D(B):
      def foo(self):
         Super(D, self).foo()
         self.supercls.foo()
         print 'D.foo'
         
      def bar(self):
         v = Super(D, self).bar()
         print v
         
      def baz(self):
         v = Super(D, self).baz()
         print v
         
   D.supercls = Super(D)
         
   d = D()
   d.foo()
   d.bar()
   d.baz()

both with and without BB as the top of the hierarchy. It seems to behave
like

  B.foo(self)
  B.bar(self)
  B.baz(self)

which I would argue is the correct behavior... (and for Super(D,
self).baz(), I'd argue it's much simpler than what you'd have to write
today <wink>).

Comments?

-- bjorn