Confused about properties, descriptors, and attributes...

[Bruno Desthuilliers]

redefined.horizons at gmail.com a écrit :
> I've been reading about Python Classes, and I'm a little confused about
> how Python stores the state of an object.

Simply put: a Python object is mostly a hashtable, with attribute names 
as keys and references to other objects as values - one of these 
name/ref pairs pointing to the class object.

> I was hoping for some help.
> 
> I realize that you can't create an empty place holder for a member
> variable of a Python object. It has to be given a value when defined,
> or set within a method.

There's a special Python object named None.

> But what is the difference between an Attribute of a Class, a
> Descriptor in a Class, and a Property in a Class?

Everything is an attribute, one way or another. Let's start simple: an 
object has a special attribute named __dict__, in which (references to) 
instance attributes - those you set with 'self.attr=val' - are stored. 
As I mentioned above, our object has another special attribute named 
__class__, that points to the class object  - which, being an object, 
has itself a __dict__  attribute in which class attributes are stored, 
and a __class__ attribute pointing to the metaclass - the class of the 
class.

Note BTW that Python functions being objects, the term 'attribute' 
include methods... (or more exactly, functions that will be turned into 
methods when looked up)

Ok, now how is this used. <overly-simplified> When a name is looked up 
on an object (via the lookup operator '.'), it is first looked up in the 
object's __dict__. If this fails, it's then looked up in the class's 
__dict__, then in parent classes __dict__s. If the attribute object 
found is a callable, it is wrapped in a method object before. If the 
object is a descriptor (and has been found in a class __dict__), then 
the __get__ method of the descriptor is called.</overly-simplified>

NB : a descriptor is an object that implements at least a __get__ method 
that takes an instance (the one on which the lookup occurs) and a class 
as params. Properties are descriptors. methods too.

The __dict__ of a class is populated with the names and functions 
defined in the class: block. The __dict__ of an instance is populated 
with the attributes bound to the instance (usually inside method, and 
mostly inside the __init__() method, which is called just after object's 
creation) - with the exception that, if the name already refers to a 
descriptor having a __set__ method, then this will take precedence.

All this is a 1000 feet high, overly simplified view of Python's object 
model - the lookup rules are a bit more complex (cf the 
__getattr__/__setattr__/__getattribute___ methods, and the complete 
definition of the dexcriptor protocol), and I didn't even talked about 
slots nor metaclasses.

To summarize: a sample class

class MyObject(object):
    classattr = "I am a class attribute"

    # the __init__ function will also be
    # an attribute of class MyObject
    def __init__(self, name):
      # _name will be an instance attribute
      self._name = name

    def _set_name(self, value):
      self._name = name
    # name will be an attribute of MyClass.
    # it's a property - which is a descriptor
    name = property(fget=lambda self: self._name, fset=_set_name)
    # we don't want to pollute the class's namespace:
    del _set_name

and some musing with it:

 >>> MyObject.__dict__.keys()
['__module__', 'name', 'classattr', '__dict__', '__weakref__', 
'__doc__', '__init__']
 >>> MyObject.__dict__['classattr']
'I am a class attribute'
 >>> MyObject.__dict__['__init__']
<function __init__ at 0x404134fc>
 >>> MyObject.__init__
<unbound method MyObject.__init__>
 >>> MyObject.name
<property object at 0x4041661c>
 >>> MyObject.name.__class__.__dict__.keys()
['fset', '__new__', '__set__', '__getattribute__', '__doc__', 'fget', 
'__get__', 'fdel', '__init__', '__delete__']
>>>> MyObject.name.__class__.__dict__['fget']
<member 'fget' of 'property' objects>
>>> MyObject.name.__class__.fget
<member 'fget' of 'property' objects>
>>> MyObject.name.fget
<function <lambda> at 0x4041356c>
 >>> MyObject.name.fset
<function _set_name at 0x40413534>
 >>> MyObject.__class__
<type 'type'>
 >>> m = MyObject('parrot')
 >>> m
<__main__.MyObject object at 0x40418dcc>
 >>> m.name
'parrot'
 >>> MyObject.name.fget(m)
'parrot'
 >>> dir(m)
['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', 
'__hash__', '__init__', '__module__', '__new__', '__reduce__', 
'__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 
'_name', 'classattr', 'name']
 >>> m.__dict__.keys()
['_name']
 >>> m.__dict__['_name']
'parrot'
 >>> m.classattr
'I am a class attribute'
 >>> MyObject.classattr
'I am a class attribute'
 >>> MyObject.classattr = 'yes I am'
 >>> m.classattr
'yes I am'
 >>> m.classattr = "and I am in fact an instance attribute shadowing 
MyObject.classattr"
 >>> m.classattr
'and I am in fact an instance attribute shadowing MyObject.classattr'
 >>> MyObject.classattr
'yes I am'
 >>> m.__dict__
{'classattr': 'and I am in fact an instance attribute shadowing 
MyObject.classattr', '_name': 'parrot'}
 >>> del m.classattr
 >>> m.classattr
'yes I am'
 >>> def sayHello(obj, who):
...     return "hello, %s, my name is %s" % (who, obj.name)
...
 >>> sayHello(m, 'Scott')
'hello, Scott, my name is parrot'
 >>> import types
 >>> m.greet = types.MethodType(sayHello, m, m.__class__)
 >>> m.greet
<bound method MyObject.sayHello of <__main__.MyObject object at 0x40418dcc>>
 >>> m.__dict__
{'greet': <bound method MyObject.sayHello of <__main__.MyObject object 
at 0x40418dcc>>, '_name': 'parrot'}
 >>> >>> m.greet('Joe')
'hello, Joe, my name is parrot'
 >>> MyObject.greet
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
AttributeError: type object 'MyObject' has no attribute 'greet'
 >>>


  If I had a Monster Class, and I wanted to give each Monster a member
> variable called ScaryFactor, how would I define it? 

class Monster(object):
   def __init__(self, scary_factor):
     self.scary_factor = scary_factor


> Does a Property
> simply provide methods that access an Attribute?

A property is mean to have simple computed attributes. The two main uses 
are to restrict access to an existing attribute (ie: make it readonly) 
or to compute something from existing attributes. Now you can use custom 
descriptors for more advanced computed attributes (that may have nothing 
to do with the object on which there are called).

HTH

> P.S. - I'm speaking about the "new" Python classes, not the "old" ones.

Hopefully. I don't see much reason to use old-style classes

> I hope my questions make sense.

It does. Python's object model is much more complex than it seems at 
first sight, and learning how to use it is worth the time spent if you 
want to really take advantage of Python's power.