[Tutor] question about descriptors

[Steven D'Aprano]

On Thu, Nov 12, 2015 at 12:11:19PM +0000, Albert-Jan Roskam wrote:

> > __getattr__() is only invoked as a fallback when the normal attribute lookup 
> > fails:
> 
> 
> Aha.. and "normal attributes" live in self.__dict__?

Not necessarily.

Attributes can live either in "slots" or the instance dict, or the class 
dict, or one of the superclass dicts. Some examples may help. Let's 
start with defining a hierarchy of classes, and make an instance:


class Grandparent(object):
    spam = "from the grandparent class"
    def __getattr__(self, name):
        return "%s calculated by __getattr__" % name

class Parent(Grandparent):
    eggs = "from the parent class"

class MyClass(Parent):
    cheese = "from the instance's own class"

instance = MyClass()
instance.tomato = "from the instance itself"


The attributes defined above return their value without calling 
__getattr__:

py> instance.tomato, instance.cheese, instance.eggs, instance.spam
('from the instance itself', "from the instance's own class", 
 'from the parent class', 'from the grandparent class')

but only "tomato" lives in the instance __dict__:

py> instance.__dict__
{'tomato': 'from the instance itself'}


You can check MyClass.__dict__, etc. to see the other class attributes. 
And, of course, __getattr__ is called for anything not found in those 
dicts:

py> instance.foo
'foo calculated by __getattr__'


Slots are an alternative to dict-based attributes. If you have millions 
of instances, all with a fixed number of attributes, using a dict for 
each one can waste a lot of memory. Using slots is a way of optimizing 
for memory:

class Slotted(object):
    __slots__ = ["spam", "eggs"]
    def __init__(self):
        self.spam = 1
        self.eggs = 2
    def __getattr__(self, name):
        return "%s calculated by __getattr__" % name

x = Slotted()


This works similarly to the above, except there is no instance dict at 
all:

py> x.spam
1
py> x.eggs
2
py> x.foo
'foo calculated by __getattr__'
py> x.__dict__
'__dict__ calculated by __getattr__'


To be honest, I didn't expect that last result. I expected it to return 
Slotted.__dict__. I'm not entirely sure why it didn't.


[...]
> > If you need to intercept every attribute lookup use __getattribute__():
> 
> Fantastic, thank you for the clear explanation. Do you happen to know 
> whether the __getattr__ vs. __getattribute__ distinction was (a) a 
> deliberate design decision or (b) a historic anomaly? 

A bit of both.

Originally, classes didn't support __getattribute__. Only __getattr__ 
existed (together with __setattr__ and __delattr__), and as you have 
seen, that is only called where the normal attribute lookup mechanism 
fails. That was deliberate.

But in Python 2.2, "new style" classes were added. For technical 
reasons, new-style classes need to support intercepting every attribute 
lookup (that provides the hook for descriptors to work). So 
__getattribute__ was added, but only for new-style classes.

But be warned: writing your own __getattribute__ method is tricky to get 
right, and tends to slow down your class. So it's best avoided, unless 
you really need it.




-- 
Steve